Feature/validator orchestrator #2057
Open
+8,047
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
KarlG-nbis
force-pushed
the
feature/validator-orchestrator
branch
from
dfbe13f to
f79cb50
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2057 +/- ##
==========================================
- Coverage 44.36% 43.87% -0.50%
==========================================
Files 49 48 -1
Lines 7001 6881 -120
==========================================
- Hits 3106 3019 -87
+ Misses 3509 3487 -22
+ Partials 386 375 -11
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
KarlG-nbis
force-pushed
the
feature/validator-orchestrator
branch
6 times, most recently
from
3a4c6fe to
c4183a2
Compare
jbygdell
reviewed
Nov 4, 2025
Collaborator
jbygdell
left a comment
jbygdell
left a comment
There was a problem hiding this comment.
There are so many thing here that goes against my normal coding principles so i don't know what to do.
More specifically:
- the use global/shared variables
- naked functions (no return values)
- functions that return functions or function wrapped values
Which is maybe why the use of the init() function never has made much sense to me. I prefer implicit calls for that kind of things as it makes for fewer gotcha moments.
Outside of that it looks like a nice proof of concept.
KarlG-nbis
force-pushed
the
feature/validator-orchestrator
branch
from
24794d1 to
2fb253d
Compare
jbygdell
previously approved these changes
Nov 5, 2025
zeidlitz
previously approved these changes
Nov 5, 2025
KarlG-nbis
force-pushed
the
feature/validator-orchestrator
branch
from
a8044dc to
4854f60
Compare
…/rbac yet), an apptainer example and kube files
… files, and override help func with func that also prints env varible name
…itive complexity with big functions
… for unit testing, add optional config for setting the work dir of validation of files
…, fix some spacing items, fix typos, add check if file paths requested was found to belong to user
…mpressed sandbox build
…not require securityContext.privileged, needing to use apptainer exec, instead of apptainer run
…add some additional api unit tests
…nd fix 100-200 words validator
…split comma separated list, do it manually
…issing file result, and readValidationInformation no file duplicates
…il response from database.ReadValidationInformation
… some func renaming, etc
… from authenticator
…available validator
… of params in db functions, and func return in api, also refactor the workers some to start on Init and add MonitorWorkers() to watch for worker errors
…ker image built, example in .kube folder, and apptainer conf changes
…elp message, set log format to json, fix gin logger
…t close broker connection, and it will close the channel
…ValidationResult rows.Close defer
…broker.Monitor and also listen to channel close events
…handling in job preparation workers on error that should not be retried, update 01_create_table_file_validation_job.sql add IF NOT EXISTS, update validators init to check if file exists before exec
… scaning, add user_id validation in admin/validate api, fix file_validation_job.finished_at column type
KarlG-nbis
force-pushed
the
feature/validator-orchestrator
branch
from
4854f60 to
1838171
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related issue(s) and PR(s)
This PR closes #990.
Description
Implementation of the sda-validator-orchestrator which is responsible for orchestrating the validation of files according to the Validator API specification
How to test