System Configuration yaml and rules + bonus rule #163
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added clear rule on consecutive runs having less time between runs than there is time in a run
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
This was referenced Jun 18, 2025
Contributor
|
@wvaske this is awesome - thanks! Do we want to indicate that Rack Units are mandatory only for certain types of systems? |
dslik
reviewed
Jun 18, 2025
Submission_guidelines.md
Outdated
| * The metric for throughput is the minimum across all GPUs. | ||
|
|
||
| Each benchmark setup must be executed five times, and logs from all five runs must be submitted. The final metrics are the average across the five runs. | ||
| Each benchmark setup must be executed five times, and logs from all five runs must be submitted. The final metrics are the average across the five runs. Runs must be consecutive with not failed runs between the selected runs. Runs can not be cherry picked from a range of runs excepting that all five runs are consecutive within the large sequence of runs. |
There was a problem hiding this comment.
Should this read "Runs must be consecutive with no failed runs ..."?
# Conflicts: # Submission_guidelines.md
…e time at the top of the Benchmark Overview
Contributor
Author
|
I merged main into the branch. It looks good to me and I think I addressed the issues here. |
johnugeorge
approved these changes
Jun 18, 2025
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added clear rule on consecutive runs having less time between runs than there is time in a run
Updated system-configuration.yaml example file
Updated rule that system-configuration.pdf needs topology