Data science and BPM

Q&A
1

Hello everyone,

Hope you all are doing well. I am an IT student specializing in a data science career, I recently read an article talking about BPM and what I would like from you people is like guiding me please. How? I would like to know different examples on how we can apply data science to BPM because I am like exploring several sectors where data science can be apply to work on it in order to grow in skills. 

Waiting to hear from you wink,

Best regards.

2

Hi Christ,

Thanks for your question. This is a hot topic in BPM and there are currently a lot of applications. I can recommend this medium article that summarizes the link between BPM and data science. But let me give you a detailed explanation. First, I will explain the type of data we consider in BPM.

Process execution data: This data comes from the execution of business processes. When processes are executed in a BPM system, the system records, for each executed activity (or step) in the process, the name of the executed activity, the time at which it was executed, the resource that executed it (it could be a human, a system, etc.) and any other associated data.

After multiple executions of the process, this data can be collected and analyzed. Now I come to: what kind of analysis we can do with this data? There are two trendy topics that answer this question: Process Mining and Augmented BPM.

Process Mining is a subfield of BPM that analyzes the process execution data with the aim of improving the process. By improving, I mean:

  • detecting and predicting performance issues (e.g. delays, bottlenecks, high resource utilization);
  • performing a root-cause analysis, and
  • recommending corrective actions and process re-design.

Most of these techniques use machine learning algorithms (e.g. neural networks, markov decision processes, and others).

I recommend looking at this LinkedIn article that illustrates the current and future applications of Process Mining.

We are also developing open source libraries for visualizing process execution data on BPMN diagrams. You can have a look at the project website and our getting started tutorial. We welcome all types of contributions on our github repository ;)

Augmented BPM complements Process Mining and looks more at the "continuous improvement" and "autonomous and adaptive BPM systems" using AI technologies. I highly recommend this LinkedIn article that explains what is Augmented BPM and how it complements Process Mining.

If you're interested in combining NLP and BPM, you can look at this medium article that explains a nice application.

I hope these pointers help you explore and discover the applications of data science in BPM. If you have any question, please don't hesitate to reach out!

Have a nice day,

Nour

 

3

Hi Christ,

Sorry for my late response. Indeed, process mining is close to data mining, in the sense that mining techniques are developed for a specific type of data which are "process execution data". Now, my question is: what do you mean by "data science"? Data science is a broad term that includes data mining, machine learning and others. Could you please clarify what data science techniques/applications you're referring to?

4

Hi Christ,

I'm not aware of such datasets that are publicly available. Did you manage to find one? If yes, I would be interested to know where you got it. 

If you didn't find yet, I think you'll need to generate an artificial one (either manually, or by finding a way to automate the generation). Maybe you can design a mission management process with Bonita that allows you to generate this kind of information. For example, the process starts with the activity "enter employee name", "enter expected expenses", "enter mission information", etc. Later on in the process, you could have the activity "mission completed?" with an exclusive gateway: if "yes", then you have the activity "enter mission duration", "enter mission expenses" and so on. You could then run this process multiple times to generate the artificial dataset with the information you entered in each run. Now the trick is that you have to be able to retrieve the information you enter when you run the process and to save it somewhere, for example in a CSV file. This is actually could be a nice example of "data collection" with a BPM solution like Bonita.

Once you have this dataset, you could do many things, for example:

1- You can imagine that there is a new run of the process for a new mission. Some activities have been executed so far, for example "enter employee name", "enter mission information". And now you want to be able to predict whether the mission will be completed or not based on the data you have collected so far from the executed activities. So in the BPM context, you want to predict which path in your process will be taken (at the exclusive gateway, is the process going to take the "yes" or the "no" branch?) You can train a neural network on your artificial data to predict such information (btw these are actually process mining related techniques).

2- In the same way, you can predict the mission duration and mission expenses (again in a BPM context, you are predicting activity related data, because remember that these are data that you enter when the activities "enter mission duration" and "enter mission expenses" are executed).

In terms of visualization, you can visualize the result of the prediction directly on your process, by for example highlighting (e.g. changing the color/width..) of the elements in your process that have been executed so far and the path that is predicted by your technique. You can do so by using our open source bpmn-visualization library. If you decide to use it and you need any help, you can contact us via the channels that appear on our website.

 

Hope this helps you & Good luck in your project! It's a nice project so don't hesitate to share it with us. 

 

 

 

5

Read this discussion. May be this can help you here https://community.bonitasoft.com/questions-and-answers/data-gosloto results-science-and-bpm. If yes then please mentioned me here.

6

Answer of: "what can be done with the generated event log":

Your log does not include "data" information. So here are some suggestions of what you can do with the log containing only information about activities: 

  1. Compute some statistics such as
    • How many times an activity is executed (frequency)
    • What is the min/max/mean duration (e.g. by computing the difference between assigned_date and last_update_date) --> this information is helpful to understand the performance of a process
    • You can look at other statistics e.g. here: https://pm4py.fit.fraunhofer.de/static/assets/api/2.3.0/pm4py.statistics.html
    • When you compute this information, you can visualize it using our libraries. This example visualizes exactly the frequency and performance information. You can customize it to visualize the information the way you want (for example by coloring activities). Our libraries allow you to customize the visualization to fit your specific needs.
  2. You can generate a dotted chart using some open source libraries such as: https://pm4py.fit.fraunhofer.de/static/assets/api/2.3.0/generated/pm4py.vis.view_dotted_chart.html 
  3. Since you have the actors who performed the activities (in executedBy), you can do organisational mining: e.g. https://pm4py.fit.fraunhofer.de/static/assets/api/2.3.0/pm4py.algo.organizational_mining.html

Advanced (your data is very small, so this is just for illustration purposes):

  • You can train a neural network to predict the remaining time of an ongoing case. You can see a case as a sequence of activities and use for e.g. LSTM. But you have to do some data transformation and to figure out how to do the feature engineering part.
7

So as I finished working on bonita community software with my mission management process, a friend of mine saw the project and wants me to deploy it in his enterprise for testing.

 

My questions goes as follows:

 

1.How can I deploy what I did in the community version in his environment?

2. I once asked how to access the portal to use the process I implemented without passing through the run button inside the software and I have been told it is by installing the bonita runtime. IS it  the case?

3. When I will install the bonita runtime how will I access the mission management process I implemented inside the bonita community version software?

8

Hi Nour,

Thanks so much for your intervention I will rapidly dive into it to understand and apply to grow my skills laugh,

Have an excellent day.

 

9

Hey Nour I read the articles you posted and I was much more interested by process mining but when going further into my discovery, I notified that process mining is instead closer or deals with data mining and not data science.

10

Hey Nour,

Sorry for responding late too. 

Let me explain. I am working on a project using a business process called: mission management process, where I have to: 
-Diagram the process with Bonita.
-Do the data collection.
-Do the data preparation
-Do the data analysis and finally
-Do data visualization.

Now I would like to implement data science on this project using machine learning but I don't know what to implement using machine learning and I would like to have some ideas from you please.

Best regards.

11

Also, I have brought out the elements that the dataset must contain since it is a mission process in particular:

name of the employee(s) for the mission
department of service of the employee(s)
start date of the mission
end date of the mission
the year in which the mission was made
name of the mission
expected duration of the mission
duration performed
mission expenses
number of employees of the mission

I don't know if you know where I can get such datasets or something similar to that please.

12

Hey Nour,

Finally for data science and BPM I will be doing process mining. What I have learned in overview is that with process mining we: Discover the process, perform diagnostics(Conformance checking and performance checking) and ameliorate the process. I would like to know if with Bonita it is possible to do process mining.

Best Regards

13

You can do process discovery with our tool: https://process-analytics.dev/model-generation-application/. Just send us your log and we'll send you back the discovered process. 

On the visualization side, you can use our open source visualization libraries (available in JavaScript and in R). To get a sense of what you can do, you can check the examples here.

All these links are accessible from the website of our project Process Analytics.

There are additional functionalities, but available only in the enterprise version.

Hope this helps!

Nour

 

 

14

How can I retrieve my logs? Please.

15

Hello still waiting your guide on how I can extract my event logs please.

Best Regards

16

There's a way for configuring the Bonita engine to log the information you need during the execution of a process (which will allow you to create your event log). But this is included only in the enterprise edition: https://documentation.bonitasoft.com/bonita/2022.2/integration/event-handlers 

What I proposed in my previous answer is like a workaround solution (which is of course not optimal, and I personally never tried it). I will detail it again here:

Solution 1:

You can create an object type in your BDM that you name for example "Event". This object has the following attributes: 

  • process-instance-id
  • activity-name
  • start-time and/or completion-time
  • any other data attributes coming from your process or bdm variables that you want to collect in your event log 

Then in your process, you instantiate this object at each activity and you fill the required attributes that you defined for the Event object.

Once you're done, you start executing your process (as much as you want). At the end, you'll have a collection of "events"  (rows in the database) coming from multiple runs of your process. All you need to do is to extract these events and store them in a CSV file so that you have an "event log". 

Solution 2:

Same as solution 1 but instead of creating an object in the BDM, you create process variables, then you write groovy scripts for each activity, that collect the required data (process-instance-id, activity name, timestamp, etc.) and store them directly in a CSV or excel file.

 

Best,

Nour

17

What about using the REST API to retrieve the required data ?

You’ll find more information on the doc site

If you want to write a client application that polls the Bonita runtime, I recommend using the bonita-java-client which is a Java client on top of the Bonita REST API.

It should be possible to design a client application retrieving the archived activities info (running once a day, it depends on the use case) and push it in a third party database like an ElasticDB for other usage (data mining…etc).

HTH
Romain

18

Hello you all,

Finally extracted the event data using the REST API technique.

Thanks for all your support.

Cheers,

19

For that I used the information from the website as I said (REST API) and the tutorial on Postman still from Bonitasoft tutorial to obtain the information. Here is the link to the tutorial: https://www.youtube.com/watch?v=DUgyEhIc3zQ

20

Hello

From my process I created a page where i display all the missions that have been planned. In my database I have 30 missions planned but on the page only the first ten display. Is it possible to increase the number? 

Best regards,