Data science and BPM
Hello everyone,
Hope you all are doing well. I am an IT student specializing in a data science career, I recently read an article talking about BPM and what I would like from you people is like guiding me please. How? I would like to know different examples on how we can apply data science to BPM because I am like exploring several sectors where data science can be apply to work on it in order to grow in skills.
Waiting to hear from you 
,
Best regards.
Hi Christ,
Thanks for your question. This is a hot topic in BPM and there are currently a lot of applications. I can recommend this medium article that summarizes the link between BPM and data science. But let me give you a detailed explanation. First, I will explain the type of data we consider in BPM.
Process execution data: This data comes from the execution of business processes. When processes are executed in a BPM system, the system records, for each executed activity (or step) in the process, the name of the executed activity, the time at which it was executed, the resource that executed it (it could be a human, a system, etc.) and any other associated data.
After multiple executions of the process, this data can be collected and analyzed. Now I come to: what kind of analysis we can do with this data? There are two trendy topics that answer this question: Process Mining and Augmented BPM.
Process Mining is a subfield of BPM that analyzes the process execution data with the aim of improving the process. By improving, I mean:
- detecting and predicting performance issues (e.g. delays, bottlenecks, high resource utilization);
- performing a root-cause analysis, and
- recommending corrective actions and process re-design.
Most of these techniques use machine learning algorithms (e.g. neural networks, markov decision processes, and others).
I recommend looking at this LinkedIn article that illustrates the current and future applications of Process Mining.
We are also developing open source libraries for visualizing process execution data on BPMN diagrams. You can have a look at the project website and our getting started tutorial. We welcome all types of contributions on our github repository ;)
Augmented BPM complements Process Mining and looks more at the "continuous improvement" and "autonomous and adaptive BPM systems" using AI technologies. I highly recommend this LinkedIn article that explains what is Augmented BPM and how it complements Process Mining.
If you're interested in combining NLP and BPM, you can look at this medium article that explains a nice application.
I hope these pointers help you explore and discover the applications of data science in BPM. If you have any question, please don't hesitate to reach out!
Have a nice day,
Nour
So as I finished working on bonita community software with my mission management process, a friend of mine saw the project and wants me to deploy it in his enterprise for testing.
My questions goes as follows:
1.How can I deploy what I did in the community version in his environment?
2. I once asked how to access the portal to use the process I implemented without passing through the run button inside the software and I have been told it is by installing the bonita runtime. IS it the case?
3. When I will install the bonita runtime how will I access the mission management process I implemented inside the bonita community version software?
Answer of: "what can be done with the generated event log":
Your log does not include "data" information. So here are some suggestions of what you can do with the log containing only information about activities:
- Compute some statistics such as
- How many times an activity is executed (frequency)
- What is the min/max/mean duration (e.g. by computing the difference between assigned_date and last_update_date) --> this information is helpful to understand the performance of a process
- You can look at other statistics e.g. here: https://pm4py.fit.fraunhofer.de/static/assets/api/2.3.0/pm4py.statistics.html
- When you compute this information, you can visualize it using our libraries. This example visualizes exactly the frequency and performance information. You can customize it to visualize the information the way you want (for example by coloring activities). Our libraries allow you to customize the visualization to fit your specific needs.
- You can generate a dotted chart using some open source libraries such as: https://pm4py.fit.fraunhofer.de/static/assets/api/2.3.0/generated/pm4py.vis.view_dotted_chart.html
- Since you have the actors who performed the activities (in executedBy), you can do organisational mining: e.g. https://pm4py.fit.fraunhofer.de/static/assets/api/2.3.0/pm4py.algo.organizational_mining.html
Advanced (your data is very small, so this is just for illustration purposes):
- You can train a neural network to predict the remaining time of an ongoing case. You can see a case as a sequence of activities and use for e.g. LSTM. But you have to do some data transformation and to figure out how to do the feature engineering part.
Read this discussion. May be this can help you here https://community.bonitasoft.com/questions-and-answers/data-gosloto results-science-and-bpm. If yes then please mentioned me here.
Hi Christ,
I'm not aware of such datasets that are publicly available. Did you manage to find one? If yes, I would be interested to know where you got it.
If you didn't find yet, I think you'll need to generate an artificial one (either manually, or by finding a way to automate the generation). Maybe you can design a mission management process with Bonita that allows you to generate this kind of information. For example, the process starts with the activity "enter employee name", "enter expected expenses", "enter mission information", etc. Later on in the process, you could have the activity "mission completed?" with an exclusive gateway: if "yes", then you have the activity "enter mission duration", "enter mission expenses" and so on. You could then run this process multiple times to generate the artificial dataset with the information you entered in each run. Now the trick is that you have to be able to retrieve the information you enter when you run the process and to save it somewhere, for example in a CSV file. This is actually could be a nice example of "data collection" with a BPM solution like Bonita.
Once you have this dataset, you could do many things, for example:
1- You can imagine that there is a new run of the process for a new mission. Some activities have been executed so far, for example "enter employee name", "enter mission information". And now you want to be able to predict whether the mission will be completed or not based on the data you have collected so far from the executed activities. So in the BPM context, you want to predict which path in your process will be taken (at the exclusive gateway, is the process going to take the "yes" or the "no" branch?) You can train a neural network on your artificial data to predict such information (btw these are actually process mining related techniques).
2- In the same way, you can predict the mission duration and mission expenses (again in a BPM context, you are predicting activity related data, because remember that these are data that you enter when the activities "enter mission duration" and "enter mission expenses" are executed).
In terms of visualization, you can visualize the result of the prediction directly on your process, by for example highlighting (e.g. changing the color/width..) of the elements in your process that have been executed so far and the path that is predicted by your technique. You can do so by using our open source bpmn-visualization library. If you decide to use it and you need any help, you can contact us via the channels that appear on our website.
Hope this helps you & Good luck in your project! It's a nice project so don't hesitate to share it with us.
Comments
Hey Nour,
Thanks again for your intervention.
I will not hesitate to share it with you when done.
Best regards.
By the way I have a preocupation right now,
1.I have already drawn my BPMN diagram and configure all what has to be done, I did it by following tutorals from the YouTube channel:Bonita Camp, but I have a problem, when an actor of the task submits a task, the next actor incharge of receiving it doesnot receive it. May you help please?
Diagramme link: https://ibb.co/y8Pbf2N
2.Also I have another question: From the Hierarchical Superior from my diagram above, the superior has to accept or deny the mission set by the mission initiator, how could I impliment that part? Possible outcomes please.
3.For email connector, during the execution of the process, does an actor write an email and send to the next actor or the email has to be pre-write and goes automatically?
Hi Christ,
Here are my answers to your questions:
1- Maybe there's something wrong with the configuration of the connectors. Did you test if they're working properly?
2- It's a human task. You can add a form to let the superior decide (approve/decline or yes/no) and store the value in a variable. This variable is then used in the condition on the exclusive gateway branches.
3- Email should be pre-write, however you can insert the values of your "variables" in the email text body by clicking on "Insert a variable" as shown in the image below. Now if you want to write an email during the execution of the process, you can let the user write the email text in a "form", store the value in a "text" variable and use this variable in your email connector message text. You should just make sure that this variable is filled before the connector is executed.
Hey Nour,
Sorry for all my questions.
Okay I want to reassure myself that I understand what you are telling me.
1.If I create the business variable that will handle the response I also have to create a table in my BDM that stores its responses too?
2. In my form I work with two buttons, one to accept and one to refuse or just one button will enough?
3.How to link the superior's answer with my Gateway?
Hi Christ,
Sorry for my late response. Indeed, process mining is close to data mining, in the sense that mining techniques are developed for a specific type of data which are "process execution data". Now, my question is: what do you mean by "data science"? Data science is a broad term that includes data mining, machine learning and others. Could you please clarify what data science techniques/applications you're referring to?
Comments
Hey Nour,
Sorry for responding late too.
Let me explain. I am working on a project using a business process called: mission management process, where I have to:
-Diagram the process with Bonita.
-Do the data collection.
-Do the data preparation
-Do the data analysis and finally
-Do data visualization.
Now I would like to implement data science on this project using machine learning but I don't know what to implement using machine learning and I would like to have some ideas from you please.
Best regards.
Also, I have brought out the elements that the dataset must contain since it is a mission process in particular:
name of the employee(s) for the mission
department of service of the employee(s)
start date of the mission
end date of the mission
the year in which the mission was made
name of the mission
expected duration of the mission
duration performed
mission expenses
number of employees of the mission
I don't know if you know where I can get such datasets or something similar to that please.
Hey Nour,
Finally for data science and BPM I will be doing process mining. What I have learned in overview is that with process mining we: Discover the process, perform diagnostics(Conformance checking and performance checking) and ameliorate the process. I would like to know if with Bonita it is possible to do process mining.
Best Regards
How can I retrieve my logs? Please.
Hello still waiting your guide on how I can extract my event logs please.
Best Regards
What about using the REST API to retrieve the required data ?
You'll find more information on the doc site
If you want to write a client application that polls the Bonita runtime, I recommend using the bonita-java-client which is a Java client on top of the Bonita REST API.
It should be possible to design a client application retrieving the archived activities info (running once a day, it depends on the use case) and push it in a third party database like an ElasticDB for other usage (data mining..etc).
HTH
Romain
Hello you all,
Finally extracted the event data using the REST API technique.
Thanks for all your support.
Cheers,
For that I used the information from the website as I said (REST API) and the tutorial on Postman still from Bonitasoft tutorial to obtain the information. Here is the link to the tutorial: https://www.youtube.com/watch?v=DUgyEhIc3zQ
Hello
From my process I created a page where i display all the missions that have been planned. In my database I have 30 missions planned but on the page only the first ten display. Is it possible to increase the number?
Best regards,
Comments
Hi Nour,
Thanks so much for your intervention I will rapidly dive into it to understand and apply to grow my skills
,
Have an excellent day.
Hey Nour I read the articles you posted and I was much more interested by process mining but when going further into my discovery, I notified that process mining is instead closer or deals with data mining and not data science.