Best practice for searches through the SearchOptionsBuilder

Q&A
1

Hello,

I need to do a search on an object that can have millions of records, and I need to use the SearchOptionBuilder.

I can build it like this: 

SearchOptionsBuilder searchArchivedOptions = new SearchOptionsBuilder(0, 1000000);

List<ArchivedFlowNodeInstance> archList =
          processApi.searchArchivedFlowNodeInstances(searchArchivedOptions.done()).getResult();

but when the second parameter of the SearchOptionsBuilder is too big then the query is very slow.

What's the best practice for doing this kind of searches?

Thanks

2

Hello,

Indeed, the pagination parameters (startIndex, maxResults) are there to limit the number of results to retrieve at once.

The good practice is to loop on the searchArchivedFlowNodeInstances() to retrieve the next page of results each time, and to aggregate, on client-side, the results in one single list if necessary. Or even better, to treat the current page of result on the fly, and query the next page, without aggregating the results in one single list.

A code sample could something like:

I would improve it that way:

int min = 0;
int max = 1000;
List<ArchivedFlowNodeInstance> archListAll = new ArrayList<>();
List<ArchivedFlowNodeInstance> archList;
do {
    archList =
            processApi.searchArchivedFlowNodeInstances(getSearchOptions(min, max).done()).getResult();
    archListAll.addAll(archList);
    min += max;
} while (!archList.isEmpty());

Where getSearchOptions() method is just a way to build a SearchOptionBuilder with your specifiy search filters, and the pagination passed as parameter.

 

Hope this helps.

3

A good page size (=maxResults) is between 100 and 1000.

4

Thanks a lot. I did this:

  int min = 0;
  int max = 1000;
  int step = 1000;
  List<ArchivedFlowNodeInstance> archListAll = new ArrayList<>();
  while (true) {
    SearchOptionsBuilder searchBuilder =
        getSearchOptions(min, max, timeStartDateMillis, timeEndDateMillis);

    List<ArchivedFlowNodeInstance> archList =
        processApi.searchArchivedFlowNodeInstances(searchBuilder.done()).getResult();

    if (archList.isEmpty()) {
      break;
    }
    archListAll.addAll(archList);
    min += step;

  }

where getSearchOptions is a method that gives me the new searchBuilder with the min and max

Any way to improve it?
Thanks

5

I would improve it that way:

int min = 0;
int max = 1000;
List<ArchivedFlowNodeInstance> archListAll = new ArrayList<>();
List<ArchivedFlowNodeInstance> archList;
do {
    archList =
            processApi.searchArchivedFlowNodeInstances(getSearchOptions(min, max, timeStartDateMillis, timeEndDateMillis).done()).getResult();
    archListAll.addAll(archList);
    min += max;
} while (!archList.isEmpty());
6

Ok, I'll modify my answer directly