Add a script to plot benchmark results #314
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This script is 100% vibe-coded. I only looked at the result, since I don't understand Python. Here is a generated summary of what this PR proposes:
Examples filtering by axis and benchmark name:
Examples with two files, filtering by axis, dark mode:
Here are the input files for the plots:
pytorch_ublkcpy_B200.json
babelstream_ublkcp_B200.json
Here are the prompts that I used:
python/scripts. Then look at the benchmark resultbabelstream_ublkcp_B200.json. Create a new scriptnvbench_plot.pyin the above directory that can read the benchmark result and plot a horizontal bar chart, where each benchmark state is plotted as a separate box. The box length is taken fromnv/cold/bw/global/utilization”python ./python/scripts/nvbench_plot.py ./babelstream_ublkcp_B200.jsonit errors, please fix the error”-awhere I can specify an axis name and value like-a T{ct}=I8. When specified only benchmark states are shown where theaxis_valuenode in the JSON has an axis namedT{ct}with avalueofI8”-aargument handling where when I enter an like-a Elements{io}[pow2]=28the[pow2]is parsed before the=and it converts the value after the=(28in this case) to 2 to the power of that value (2^28in this case)”-bwhere i can select the benchmark name”Device=Npart from the bar label”The get_cmap function was deprecated in Matplotlib 3.7 and will be removed in 3.11. Use
matplotlib.colormaps[name]ormatplotlib.colormaps.get_cmap()orpyplot.get_cmap()instead.Please fix it”
14. “Please reduce the empty space above the first bar and below the last bar”
15. “Please write a pull request description about what this new script can do”
16. "Add --dark mode with black background and white text."
17. "Ensure title is visible in dark mode."
18. "Remove the x‑axis description label."
19. "If -a is used, remove those axis tokens from labels and add them to the title."
20. “Please read through the nvbench code here and figure out whether it supports -a paramters where we can also specify multiple values”
21. “Please implement this”
22. “If multiple values are specified for an axis, retain the axis value in the box label”