Allow aggregations

[rain2o]

While trying to use SearchQuery to build queries for custom entities, it does not provide a way for me to add aggregations to the queries. Previously when using bodyBuilder we were requesting aggregations on certain queries for custom entity types using query = query.aggregation('terms', 'ratings.value') for example (for reviews/ratings in this example). However, I can't accomplish this aggregation using SearchQuery.

I see that aggregations are added in the core code of SearchQuery to add aggregations under a very specific condition (specifically must be in 'catalog' scope), but there is no way to add our own aggregation requests.

[cewald]

You could add a custom filter to your API.

For example, add a filter like:

import { FilterInterface } from 'storefront-query-builder'

const filter: FilterInterface = {
  priority: 1,
  check: ({ attribute }) => attribute === 'aggregation',
  filter: ({ value, queryChain }) => {
    return queryChain.aggregation(
      value.type,
      value.field
    )
  }
}

export default filter

This way you could send a query like:

{
    "_availableFilters": [ ],
    "_appliedFilters": [
        {
            "attribute": "aggregation",
            "value": {
                "type": "_max_score",
                "field": "custom_field"
            },
            "scope": "default"
        }
    ],
    "_appliedSort": [],
    "_searchText": ""
}

This should result in a ES query like:

{
  "aggs": {
    "agg__max_score_custom_field": {
      "_max_score": {
        "field": "custom_field"
      }
    }
  }
}

I can't find a documentation for custom-filters – I'll search for the PR where I added it to vue-storefront-api and storefront-api. – #6

[rain2o]

Thanks for the info @cewald . Having documentation for this would help, as I didn't know this was possible, and I'm having trouble figuring out how to implement this.

For my current need, I'm using the api-search-query core search adapter for the search functionality. I have added a custom search adapter which only registers some new types, but otherwise I would prefer not to override the search function. The search I'm currently trying to aggregate is for review, which is actually registered in the core types (though I'll probably have this need for custom types as well).

I see in the api search adapter it uses the buildQueryBodyFromSearchQuery function which I see now takes the custom filters option, but the api-search-query adapter does not. So in order to use this approach does this mean I will need to either go back to using the api search adapter, or customize the search function in my api-search-query adapter? I'm not seeing another way to add my custom filter.

In long term, having the ability to add aggregation directly while building the search query would still be ideal.

[cewald]

Yes indeed, this clearly lacks in documentation. The custom-filters are currently only implemented in the /catalog requests, but could easily be integrated at the other endpoints.

Here is a short description how to add a custom for the storefront-api:

• Create a new catalog-extension in packages/default-catalog/api/extensions/, we will use the sample-module in this folder called /example-custom-filter
• Add this module to your configs like:

  {
    ...
    "modules": {
      "defaultCatalog": {
        "registeredExtensions": [
          "icmaa-catalog"
        ]
      },
      ...
    }
    ...
  }
  

• Create your custom-filter to mutate your search-query in your modules folder under filter/catalog/SampleFilter.ts:

  import { FilterInterface } from 'storefront-query-builder'
  
  const filter: FilterInterface = {
    priority: 1,
    check: ({ operator, value, attribute, queryChain }) => operator === 'loremIpsum',
    filter ({ value, attribute, operator, queryChain }) {
      // Do you custom filter logic like (see bodybuilder.js documentation):
      // queryChain.filter('terms', attribute, value)
      return queryChain
    },
    mutator: (value) => value[Object.keys(value)[0]]
  }
  
  export default filter

• Add this custom filter to your configs:

  {
    ...
    "extensions": {
      "example-custom-filter": {
        "catalogFilter": [ "SampleFilter" ],
      },
      ...
    }
  ...
  }
  

• Now you can use it like the following in your queries:

  {
      "_availableFilters": [ ],
      "_appliedFilters": [
          {
              "attribute": "your-attribute",
              "value": {
                  "loremIpsum": "your-search-criteria"
              },
              "scope": "default"
          }
      ],
      "_appliedSort": [],
      "_searchText": ""
  }
  

That should be it, I hope it helps for now. But I agree with you, the API documentation should be updated and this feature probably could be added to the other ES endpoints as well.

[rain2o]

@cewald Ok, I see what I was doing wrong. I missed your first mention of "You could add a custom filter to your API.", I was trying to add this custom filter in Vue Storefront, not in the API layer. It's much more clear now, and I was able to add a custom filter.

This is still quite a round-about way to allow aggregations. It seems strange to need to add a custom filter to apply an aggregation, as those are two separate things within an ES query. It would make more sense to directly add an aggregation while building the query on the front-end as opposed to adding a filter and then creating a custom filter to change the filter to an aggregation on the API layer.

[cewald]

This might seem a bit sophisticated, but if you would add an aggregation directly in the frontends query, the VSF wouldn't be agnostic anymore. One benefit of the storefront-query-builder imo is to let the query-language in the frontend be independent from your underlying database-source. But yes, the second you are heading for an aggregations (like in the /catalog request for filter-navigation) you're already not agnostic anymore. 💁🏻

[rain2o]

Hmm, I can sort of see what you mean about aggregations not being agnostic. I guess I see VSF as being agnostic as a framework, but each implementation of it will include platform-specific code within themes and src modules. In this example, if I didn't add the aggregations, then I would need to make multiple requests to acquire the information I need, which would be much less performant than a single request with aggregations.

I am having trouble seeing the difference in being "agnostic" of the database layer between requesting data and using the values of that data vs requesting an aggregation of said data. It's requesting and using the same data/fields, just requesting a summary of it vs a list of the data.

And as you said, to accomplish this aggregation, you still need to add a filter on the front-end for the aggregation, and then create the custom filter on the API layer, so in this respect the frontend is still aware of the aggregation request.