Starting a process based on external data

Q&A
1

Hey folks,

I can start a process in many ways using bonita/server. One part I am trying to understand, is how to start a process based on an external database system?

I have a database used for my HR systems on a local server. Same location as the bonitaserver.

When a new employee is created in the HR application/database, I want bonita to run a process.
The program I use for HR system cannot send a message to bonita to start the process. All the program can do is what it was built to do - add a new staff member in to its own database.

How can I set bonita to see that it needs to start a process based on the new record?

I can set a process to check the database every ‘x’ seconds or minutes for new records, but that constant ongoing check is a waste of resources for the odd time when a new person is available to run a process with…

How do you do it? Really, id like bonita to know to start a process when the data changes but to not have to contantly check the database.
The other idea I had was to have HR select the new person in bonita to start the process manually. But then, this is not entirely automatic.

Thoughts?

Many thanks,
Jim
Jim

2

Anybody?

3

Hello,
In general, you have two options: a process that will check regularly if there is anything new in the external application or external application triggering a process. If you say that the external application cannot do so, then you can only create a process and run it regularly. It should be very much time-consuming if you have such a process executed every n minutes. It would be a very simple process in case there is nothing new - one task with one connector, for example. If you can have a hold on the HR database, you might create a database trigger, but I am not sure what impacts it might have on your HR application.
You can also have a small cron job outside of Bonita and of your HR application, that would regularly check for new records and notify Bonita in order to create a new instance.
I hope this can help you,
Haris

4

Hi Jim,

making sense yes, boy what a project…reminds me a little of the container shipping yard simulations I used to run way back when…

It would be nice to work on something like this again.

Here is one idea/issue that is a must for your project, and one that must not be dismissed, so remind everyone continuously throughout the project…

It’s called the X-factor. NO, not that god awful program run by Simon Cowell. But the unknown X-factor that despite everything working will bite you. I’ll come back to this.

I can see a couple of thoughts already…

“can have one process each day at midnight that looks at all the medicine table, gets the schedule for when each type needs to be given”

where does the schedule come from the medicine table or the patient table? Doctors may prescribe the medicines at different times for different patients. It may be what you’re thinking, but it’s not how it’s written, it reads the schedule is attached to the medicine…probably not what you want, and there is a good reason…

So you run your schedule and get all the medicines ready but a patient discharges themselves…for whatever reason. The giving of the medicine is still in the system…a potential security issue as the medicines could be exposed to wrongful use. And this is one of the X-factors. Human-Nature, someone may take advantage of loose meds being available. Obvioulsy there will be a discharge mechanism but will it operate to quickly or too slowly?

Then the other X-factor, bathroom breaks for example, you say start a process 5 minutes before the medicine is needed for each person/schedule. Wow, no-human will be that strict in timing…it doesn’t happen, the X-factor.

There are multiple ways of looking at your project and I’m sure the BA’s will come up with something. But one thing is for sure, management will have to accept some “wastage” to keep things going. Haris said it earlier, a cron job, or other scheduled job may be best, purely for sanity reasons…it’s simple and it works.

Final thought, you have limited resources (staff), limited beds, unlimited day-care patients, and unlimited medicines and checks…this isn’t necessarily about giving the medicines out, it’s more about enabling those limited resources to do everything within a limited amount of time (and in some cases to a fixed schedule).

Your project is not necessarily a process driven one, it is a schedule one, that is it’s a continuously changing project plan.

Try not go get the scheduling of the business mixed up with the processes…even though the scheduling is a process…

Then again, this is such a specialized requirement, and is potentially compliance ridden with issues, why not go for a ready made package?

Yes the initial cost may be higher, but with a project to run a care home where compliance with regulation are a must it might be easier and cheaper in the long run.

Have a look at:

Compliance Management

and the best one I’ve seen - including medication management is iCareHealth. At least talk to them to find out what they can do and either use them as a basis of your ideas, or use their system…

Disclosure: I have nothingt o do with the noted companies and accept no responsibilities etc…

best regards
Seán

5

First things first, get off H2…

It’s great for development and testing but not for what it looks like you are doing, real production work. Definitely not the way to go.

You’re quite specific about what you want which is good and all the answers given are good, but for what you’re describing I would go with Triggers, see here and here… for an intro. Bonita can’t help you with specifics regarding other vendors products…

In this case I would create a trigger that fires off a REST call to your process with a HRid of the record you want to process, the Bonita process can then read the HR database as required by connector etc.

Web REST infor here…

Hope this helps, regards
Seán

6

PS…

you say…

That is ok for now as we have just the one process, but in 2 years, when I have thousands based on many data that could change, checking a database(s) every ‘x’ minutes per process would take a lot of our resources.

If it does go from 1 to several thousand processes on much data that changes you might want to “Business Analysis” your whole requirement rather than “bolting on”, it looks very significant and my previous statement of move to another database for example Postgres (for example) is strongly recommended…

regards
Seán

7

Hi Sean,
Yes - we are just about to go up a subscription version and will be using a production environment at that point.

I am still in the dev phase/sort of so doing/learning what I can until the live is available :slight_smile:

We have BA’s that will be working on this, but even so, we expect to have many processes which can be run on demand.

For example, we look after patients. At different times, different patients require different medicine.

We will build a database of patients, medicine, schedules of when the medicine is needed. So on so forth. In this example, I can have one process each day at midnight that looks at all the medicine table, gets the schedule for when each type needs to be given, and then starts a sub process/process with each instance to then assign to the worker later in the day shortly before the medicine is needed.

Now thats 1 process to the database to get the list to run for today and ‘n’ databases calls depending on how many were scheduled. - thats not bad at all.

But we dont just give medicine, as a care provider you can imagine how many hundreds/thousands of checks that we may need to do per day/week/month/year for each person we care for.

If each check returned no ‘work’, its wasted process/resources.

It would be far better if the database was smart enough to be able to call our bonita server/API to start a process 5 minutes before the medicine is needed for each person/schedule. Then a process runs as work is needed, not because work could be needed saving resource.

Am I crazy? Please tell me im at least making sense :slight_smile:

8

“Now thats 1 process to the database to get the list to run for today and ‘n’ databases calls depending on how many were scheduled. - thats not bad at all.”

Let me explain what I mean by this. The 1 database call would be the: ‘select id from medicinetogive where required today’ sql call.
This would return a list of id’s if medicine is needed today. The next process step would start a ‘schedule giving medicine’ process for each id in the list.

The ‘schedule giving medicine’ process will have an id passed over. It will then get the metadata from the database (what medicine, when, how much, who) and pause/wait until the medicine should be given.

So, 1 call to the database resulting in n calls based on how much needs doing.

This can of course be improved I am sure, so all ideas welcome…

9

Hello Haris,
You are very helpful. Very much appreciated.

I have a process setup to check for new data every 15 minutes. Mostly, no changes are found. But, the request was still made which used resource/processing power. That is ok for now as we have just the one process, but in 2 years, when I have thousands based on many data that could change, checking a database(s) every ‘x’ minutes per process whould take a lot of our resources.

I had considered the cron job but it would still be a ‘thing’ regularly checking a database at set intervals and using resources.

The ideal for us is for the database to tell bonita, on demand, to trigger a process as certain data we set to be the trigger has changed. No wasted resources as work is started when it needs to start as ‘work’ is available to do. Also, as it is on demand, it is more instant.

Could you give me more information on the triggers? I have an ‘HR’ SQL server and a seperate Bonita H2 server using windows server 2008r2. Would the HR database server be able to run a trigger on save which will instantly start a bonita process? Or am I dreaming :slight_smile:

With the trigger, would bonita still have to be set to constantly ‘check’ the SQL server?

10

It would be a DML trigger on your HR database that calls an external program. You can Google out more about it, here is an example for MS SQL Server. Be aware that there are some drawbacks to this approach (if you have a rollback in your transaction, your call to external application would still be done etc.). I don’t think it is the best approach, personnaly. I would still go for a regular check on your HR database. One select should not cost too much on your database server, even if you do it “thousands of times every 15 minutes”. If you do not want to overload Bonita BPM with that, than a cron might be the best compromise.

11

Hi Sean,

The schedule would be in neither table, it would be in a middle table linking the ‘medicine_table’ and ‘patients_table’. So a ‘schedule_table’ if you will. This is because of the M:M relationship between medicine and patients.

We have looked at other products but they do not meet our needs. We have requirements to be very specific in what and how we work. Bonita fits this quite well as we can be very exact in how our process/schedules work and design them around us rather than trying to adapt other solutions which only partly meed our needs. For example, we looked at a product where you can schedule medicine. Imagine I am required to have medicine at 11am. Imagine it was given at 11:15am. The next day, the product we looked at would have switched the default time for that person to 11:15am… not good for us. The vendor also said that setting cannot be changed. - for this reason, we need control over everything; bonita gives us this as we can change our processes to meet out needs exact. - I forget the vendor so cannot type the name.

We would have a process should medicine need to be given at a random time on the fly ‘Give Medicine Process’.
But the majority of medicine, idealy, should be planned. So it would be in the schedule table.

The times in the schedule would be for when the patients are available. Due to how we work, we can plan this to be quite exact. Yes, if a staff member is unavailable, the task could run over - but we would store as KPI the start time of the task, when it was assigned, when they accessed the task and when they completed the task. So, we know metrics of what happened and when.

We go beyond giving medicine and care for people entirely. Planning when to take them to school, church, sports, so on so forth. Our ‘schedule’ is being developed so that if a task time conflicts, it will trigger a change which allows us to be somewhat dynamic. It is a really cool project.

The ‘X Factor’ is something to keep in mind at every stage, I agree completely. All we can do though is to try and match the requirements a close as possible. I dont define them, I have to make bonita meet them. Should an x factor be overlooked, it would be due to the team that defined them and thats above my grade. If something looks obviously wrong i’d raise it, but they know far more about the processes they require than me so I cannot really go anywhere…

So, I think the best route to go forward is for our processes to check frequently on the current data/schedule. It is a build up of resources on the network/servers, but it puts the control centrally on bonita. If a check runs bonita can decide what to do/what processes to start based on the data rather than hoping the other multiple data sources notify bonita/are working/are up.

A cron job outside of bonita saves the resources on bonita as it is then more on demand. That would make sense, but thinking as above we would lose the central control.
I think it would be best to keep in bonita.

Thank you for your help Sean,
Are you based in France? I am looking for some contacts in the UK…

Jim

12

Hi Jim,

Good stuff, I like the comment - above my (pay) grade…

Based in France - nope - I like you am a technician/developer/BA/CEO of my own company and while it is based in UK and Gibraltar - I actually live in Saudi Arabia (for now). :wink:

I’m hoping my project will go to market soetime soon, but there are niggling problems…

regards
Seán

13

Thats no problem at all.

‘but there are niggling problems…’ - what sort of problems are you having? I’d like to suggest ideas if I could :slight_smile:

Thanks,
Jim

14

Niggling problems are not with Bonita as such but of my own devices…

for example using the wrong call…see here .

At one time using startProcess, then trying to use sendMessage, then going back to startProcess with variable setting on next process (basic design principles really).

Developing complex, looping code to execute multiple new processes, assigning these to multiple individuals, and trying to get them to work correctly with documents…

Personally - I do Agile, but Agile is as Agile isn’t…there can be too many options on how to design and implement, which means, code rewrites are frequent.

Oh and of course there is the wife, nag, nag nag nag nag…

There there is JavaScript on the client end, for some things…learning to write JavaScript is fun…

And then there is the whole thing of having to learn java…I grew up on IBM PL/S…

All good fun. :slight_smile:

15

‘At one time using startProcess, then trying to use sendMessage, then going back to startProcess with variable setting on next process (basic design principles really).’

^ So, sort of a recursive process? It starts, gets a variable based on something and then passes the variable to its self to start again, but with the variable?

16

No - that’s just the use of the various (and multitudinous) ways of starting a process, did it one way, then had to change it to something else, then to a third requirement :slight_smile:

Flexibility comes to mind…