I am trying to implement the inbuilt pagination feature available in elastic search composite aggregation on aggregated data. This works perfectly fine until we are not removing certain buckets based on conditions of aggregated fields. To remove buckets I am using BucketSelectorAggregation.

I understand that composite aggregation doesn't support pipeline aggregation fully and the reason for the issue is after_key and size always consider the aggregation response for pagination, rather than filtred aggregated data.

If anybody can help on this to apply the pagination on the aggregated filtered bucket using some kind of filter or other option without using pipeline aggregation that would be great.

Example :

id sum

1 100
2 200
3 400
4 500
5 100

Query after aggregation to get 2 records

Result

id sum

1 100
2 200

Query after aggregation and BucketSelectorAggregation to get 2 records where sum value 100

Result

1 100

Expected Result

1 100
5 100