Transitive Searches

[whatisgalen]

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Description of Change

This PR tries to address the question of: Now that the Resource Relations index is removed, how can we make related resources searchable in Elasticsearch (beyond strictly resource-instance tile advanced search)? To answer the question, it proposes a new search-view such that a user can effectively SELECT related instances of a search result set.

This PR proposes adding new index mapping fields:

• fromrelations: a nested field that stores unidirectional pointers to related resources from the document resource, along with a few properties from the resourcexresource model instance.
• torelations_graphids: a flat array of dinstinct graphids indicating whether any other resource of those graphids has a relation to the document resource

What's the use case?

If I want to find instances of model X such that they are related to the result set of instances of model Y via search filters A, B, and C.

Pseudo bidirectionality

It's not feasible to store all the relations to a given resource, so storing just the graphid accomplishes the minimum of letting you filter in ES for those resources that have any relations to particular Resource Model instances vs those that don't, limiting the scope of the subsequent ORM query to the ResourceXResource model. That query looks like this in transitive_search_view.py:

for hit in response_object["results"]["hits"]["hits"]:
            query_ids.extend(
                rel["resourceid"]
                for rel in hit["_source"]["fromrelations"]
                if rel["graphid"] in related_graphids
            )
            if len(
                list(
                    filter(
                        lambda x: x in related_graphids,
                        hit["_source"]["torelations_graphids"],
                    )
                )
            ):  # create the list for the first Q
                toresource_ids.append(hit["_id"])

query_ids.extend(
            [
                str(rxr.from_resource_id)
                for rxr in ResourceXResource.objects.filter(
                    Q(to_resource_id__in=toresource_ids)
                    & Q(from_resource_graph_id__in=related_graphids)
                )
            ]
        )

Issues Solved

Addresses in part #10644 but does not close it as more search infrastructure is needed.

Checklist

• I targeted one of these branches:
  • dev/8.0.x (under development): features, bugfixes not covered below
  • dev/7.6.x (main support): regressions, crashing bugs, security issues, major bugs in new features
  • dev/6.2.x (extended support): major security issues, data loss issues
• I added a changelog in arches/releases
• I submitted a PR to arches-docs (if appropriate)
• Unit tests pass locally with my changes
• I added tests that prove my fix is effective or that my feature works
• My test fails on the target branch

Accessibility Checklist

Developer Guide

Topic	Changed	Retested
Color contrast
Form fields
Headings
Links
Keyboard
Responsive Design
HTML validation
Screen reader

Ticket Background

• Sponsored by:
• Found by: @
• Tested by: @
• Designed by: @

Further comments