Skip to content

Conversation

@epugh
Copy link
Member

@epugh epugh commented Apr 11, 2025

Description

Once a Case is loaded, we snapshot it. We may NOT have scores. But maybe that is okay? I did the quick version where after the docs are loaded we snapshot it. This is happening BEFORE the scorer runs.

@david-fisher how important is we store the scores? Historically we would store a score per query, but then, we would recalcluate everything (I think) when you would do a snapshot compare. I think the score in the snapshot_query isn't useful because we want to move to a future with multiple scorers being used!

I tried it with out 5000 queries case, and the snapshot JSON object was so big teh NGINX server puked with a 429 entity too large. But with 400 queries worked just fine.

Motivation and Context

As raised by @david-fisher in #1310 we should really think about why we aren't snapshotting every try.

How Has This Been Tested?

Screenshots or GIFs (if appropriate):

Types of changes

  • [] Bug fix (non-breaking change which fixes an issue)
  • [] Improvement (non-breaking change which improves existing functionality)
  • [] New feature (non-breaking change which adds new functionality)
  • [] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • [] My code follows the code style of this project.
  • [] My change requires a change to the documentation.
  • [] I have updated the documentation accordingly.
  • [] I have read the CONTRIBUTING document.
  • [] I have added tests to cover my changes.
  • [] All new and existing tests passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants