Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
118 commits
Select commit Hold shift + click to select a range
455678e
save each test generator
tjfulle Oct 31, 2025
0fa40ca
quick commit
tjfulle Nov 2, 2025
e64fbd2
another update
tjfulle Nov 3, 2025
da3961d
no more canary exec, use canary run
tjfulle Nov 3, 2025
4667fca
rempove broke import
tjfulle Nov 3, 2025
8b06d97
use stage instead of lock
tjfulle Nov 3, 2025
29d6ab7
add fairly comprehensive tutorial document for the prototype workflow
mdmosby Nov 3, 2025
742dc5e
more work with tagging/staging adding gc
tjfulle Nov 3, 2025
98b315d
create markdown reports
tjfulle Nov 4, 2025
56781d2
html and json reports
tjfulle Nov 4, 2025
d95d04b
finish up some reporting
tjfulle Nov 4, 2025
82d53f5
removing usage of session in favor of repo
tjfulle Nov 4, 2025
fc9fda8
change name repo->workspace. make tag objects lightweight. log and …
tjfulle Nov 5, 2025
f23b39d
Update gc.py
tjfulle Nov 5, 2025
2b5e466
draft refined workspace command spec
mdmosby Nov 5, 2025
34fa028
some clean up to workspace code (better error messsages and predicate…
tjfulle Nov 5, 2025
6ad3376
fixing status and hpc run/exec
tjfulle Nov 5, 2025
6028875
Can rerun failed!
tjfulle Nov 5, 2025
42a6f4c
dont tag unless explicitly asked to do so.
tjfulle Nov 5, 2025
7f6a95f
add tag -d to remove tags and fix some bugs
tjfulle Nov 6, 2025
0a615e2
config:view option lets one change the name of the view, or not gener…
tjfulle Nov 6, 2025
8fd88ca
Better garbage collecting
tjfulle Nov 6, 2025
83e8dd9
tests pass!
tjfulle Nov 6, 2025
95a8b14
cmake tests pass
tjfulle Nov 6, 2025
a3ce417
fix canary hpc tests
tjfulle Nov 7, 2025
2fd24ad
combine hpc and batch commands - batch is an alias for hpc
tjfulle Nov 7, 2025
de7bf0e
fix show excluded tests
tjfulle Nov 7, 2025
74b9105
Update workspace.py
tjfulle Nov 7, 2025
bd425c7
Update select.py
tjfulle Nov 7, 2025
cc199dc
Update workspace.py
tjfulle Nov 7, 2025
ead5362
first pass at test specs
tjfulle Nov 8, 2025
843e2b5
spec commons
tjfulle Nov 9, 2025
ad9b0af
DraftSpec -> ResolvedSpec -> TestSpec
tjfulle Nov 9, 2025
113a9ef
adding new process pool executor
tjfulle Nov 10, 2025
c7bf0dd
can run pyt files with new process pool
tjfulle Nov 10, 2025
0817053
more work on test specs
tjfulle Nov 11, 2025
4dbe10e
another update
tjfulle Nov 11, 2025
0f30fbc
fix codes put on tests
tjfulle Nov 11, 2025
1004e3f
some more linting
tjfulle Nov 12, 2025
0f133d9
another update
tjfulle Nov 12, 2025
7dc75fb
more type checking
tjfulle Nov 12, 2025
cd7ec5c
almost cmake
tjfulle Nov 13, 2025
e520915
cmake works and running from file in test session works
tjfulle Nov 13, 2025
2bd0f17
run batches
tjfulle Nov 13, 2025
27d874f
fix instantiation of execpace
tjfulle Nov 13, 2025
16b660a
run in a current session
tjfulle Nov 13, 2025
aa572c0
fix some typing error
tjfulle Nov 13, 2025
0b43f9d
reenable rebaseline
tjfulle Nov 14, 2025
7616876
more robust pool processing
tjfulle Nov 14, 2025
eda9062
fixing some more subcommands
tjfulle Nov 14, 2025
8228e54
get ty check to work on src. still not on tests
tjfulle Nov 14, 2025
90620b3
one more
tjfulle Nov 15, 2025
08afde9
resourcequeueexecutor
tjfulle Nov 15, 2025
3b47d57
ruff
tjfulle Nov 15, 2025
f0e9a6d
remove some config methods
tjfulle Nov 15, 2025
abdc5c2
config is a simple dict
tjfulle Nov 15, 2025
0b84e27
add scratch section
tjfulle Nov 15, 2025
00eeb63
better config funciton name: update -> write-new
tjfulle Nov 15, 2025
b9b5c3b
ruff
tjfulle Nov 15, 2025
14d8807
Merge branch 'main' into tjfulle/specs
tjfulle Nov 15, 2025
5b70fc9
simplify config to not have config: key
tjfulle Nov 15, 2025
06a779f
toward getting tests running
tjfulle Nov 15, 2025
cab2a47
some more work on signals
tjfulle Nov 16, 2025
d2f9015
JobProtocol
tjfulle Nov 17, 2025
36efc53
plugin batching to newer signal processing
tjfulle Nov 17, 2025
8d4f5d9
some cleanup
tjfulle Nov 17, 2025
fd9b2d9
adding protocols file and refactor resource queue
tjfulle Nov 17, 2025
a5128e4
Update queue.py
tjfulle Nov 17, 2025
ae233ca
clean up queue a bit
tjfulle Nov 17, 2025
acf0518
move batching to conductor
tjfulle Nov 17, 2025
91370d5
add batchjob module
tjfulle Nov 17, 2025
ad0e5dc
stub
tjfulle Nov 17, 2025
72f035e
one more
tjfulle Nov 17, 2025
d8bc9b7
one more
tjfulle Nov 17, 2025
dbd89bd
Adding batch executor
tjfulle Nov 18, 2025
81ec6c2
Update batchexec.py
tjfulle Nov 18, 2025
bc02edd
Update batchexec.py
tjfulle Nov 18, 2025
4174ce7
Update json_helper.py
tjfulle Nov 18, 2025
7be046b
Update config.py
tjfulle Nov 18, 2025
ab9f3ea
Update queue.py
tjfulle Nov 18, 2025
808c193
High level subprocess wrapper to print exc
tjfulle Nov 18, 2025
c5a9f8d
better signal handling in batch runner
tjfulle Nov 18, 2025
713658b
Adding command attribute when case is run
tjfulle Nov 18, 2025
66c853a
add plugin hooks earlier in the call as part of config setup. batch …
tjfulle Nov 18, 2025
3e2c75a
cleanup
tjfulle Nov 18, 2025
8760914
fix how resources are checked out and add a set_status method to test…
tjfulle Nov 18, 2025
ee09b15
core tests fixed!
tjfulle Nov 19, 2025
0e30a16
fix vvtest unit tests
tjfulle Nov 19, 2025
8bb62d5
`log`: fix FileNotFoundError logic
mdmosby Nov 19, 2025
9c96c04
`describe`: get display_name from case (callable), not spec (property)
mdmosby Nov 19, 2025
e300b71
cmake tests pass
tjfulle Nov 19, 2025
f55d985
fixed canary_hpc tests
tjfulle Nov 19, 2025
4452f28
fix tar/compress for pushing to cdash
tjfulle Nov 19, 2025
f854a30
make refresh robust
tjfulle Nov 19, 2025
bfc66e8
do not execute tests that have status other than "READY"
mdmosby Nov 19, 2025
f494840
capture/log child process exceptions and mark the status as ERROR
mdmosby Nov 19, 2025
7a7a2e1
improve error reporting for `canary init` when the workspace already …
mdmosby Nov 19, 2025
1a51b98
more robustly remove the workspace during setup
mdmosby Nov 19, 2025
76516ba
add testspec_modify and cdash artifacts bundled as one
tjfulle Nov 19, 2025
43d08e1
give with_traceback a concrete function interface
tjfulle Nov 19, 2025
7315822
fix bug in with_traceback that didnt pass kwargs through
tjfulle Nov 20, 2025
23ab0fd
adding in create_scope config method for backward compatibility
tjfulle Nov 20, 2025
a4e4370
adding some more logging
tjfulle Nov 20, 2025
63677ed
adding yet more logging
tjfulle Nov 20, 2025
b98f4f1
adding yet more logging
tjfulle Nov 20, 2025
22121a5
fix collectreport to only take specs
tjfulle Nov 20, 2025
7cf67b3
add keywords to testcase
tjfulle Nov 20, 2025
f453c13
speed up testspec resolve a little
tjfulle Nov 20, 2025
755540c
fixes:
tjfulle Nov 20, 2025
91691c1
fixing warnings found by bandit
tjfulle Nov 20, 2025
a6bc56a
ruff format/check
tjfulle Nov 20, 2025
d5ff2d9
Limiting default number of workers
tjfulle Nov 20, 2025
c7f6959
cleaning up some logging
tjfulle Nov 20, 2025
a66238b
fix incorrect log message
tjfulle Nov 20, 2025
0e8df33
clear out queue if cancelled
tjfulle Nov 20, 2025
50b11cc
re-raise keyboard interrupt
tjfulle Nov 20, 2025
5e1fe33
fixing vvtest tests
tjfulle Nov 21, 2025
c720073
ruff docs
tjfulle Nov 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 10 additions & 14 deletions docker/flux/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,8 @@ echo " "
exit_code=0
canary -d run --show-excluded-tests -w -b scheduler=flux ./examples || exit_code=$?
if [ "${exit_code}" -ne 30 ]; then
cat TestResults/.canary/config || true
cat TestResults/.canary/batches/*/*/resource_pool.json || true
cat TestResults/.canary/batches/*/*/canary-out.txt || true
cat .canary/cache/canary_hpc/batches/*/*/resource_pool.json || true
cat .canary/cache/canary_hpc/batches/*/*/canary-out.txt || true
cat TestResults/basic/second/second/canary-out.txt || true
cat TestResults/basic/second/second/canary-err.txt || true
exit 1
Expand All @@ -47,9 +46,8 @@ echo " "
exit_code=0
canary -d run --show-excluded-tests -w -b scheduler=flux -b spec=count:3 ./examples || exit_code=$?
if [ "${exit_code}" -ne 30 ]; then
cat TestResults/.canary/config || true
cat TestResults/.canary/batches/*/*/resource_pool.json || true
cat TestResults/.canary/batches/*/*/canary-out.txt || true
cat .canary/cache/canary_hpc/batches/*/*/resource_pool.json || true
cat .canary/cache/canary_hpc/batches/*/*/canary-out.txt || true
cat TestResults/basic/second/second/canary-out.txt || true
cat TestResults/basic/second/second/canary-err.txt || true
exit 1
Expand All @@ -62,9 +60,8 @@ echo " "
exit_code=0
canary -d run --show-excluded-tests -w -b scheduler=flux -b spec=count:3,layout:atomic ./examples || exit_code=$?
if [ "${exit_code}" -ne 30 ]; then
cat TestResults/.canary/config || true
cat TestResults/.canary/batches/*/*/resource_pool.json || true
cat TestResults/.canary/batches/*/*/canary-out.txt || true
cat .canary/cache/canary_hpc/batches/*/*/resource_pool.json || true
cat .canary/cache/canary_hpc/batches/*/*/canary-out.txt || true
cat TestResults/basic/second/second/canary-out.txt || true
cat TestResults/basic/second/second/canary-err.txt || true
exit 1
Expand All @@ -77,9 +74,8 @@ echo " "
exit_code=0
canary -d run --show-excluded-tests -w -b scheduler=flux -b spec=count:auto,layout:flat ./examples || exit_code=$?
if [ "${exit_code}" -ne 30 ]; then
cat TestResults/.canary/config || true
cat TestResults/.canary/batches/*/*/resource_pool.json || true
cat TestResults/.canary/batches/*/*/canary-out.txt || true
cat .canary/cache/canary_hpc/batches/*/*/resource_pool.json || true
cat .canary/cache/canary_hpc/batches/*/*/canary-out.txt || true
cat TestResults/basic/second/second/canary-out.txt || true
cat TestResults/basic/second/second/canary-err.txt || true
exit 1
Expand All @@ -89,5 +85,5 @@ echo " "
echo " "
echo "----------------------- Done! ----------------------"
# Artifacts
canary -C TestResults report junit create -o $CI_PROJECT_DIR/junit.xml || true
canary -C TestResults report cdash create -d $CI_PROJECT_DIR/xml || true
canary report junit create -o $CI_PROJECT_DIR/junit.xml || true
canary report cdash create -d $CI_PROJECT_DIR/xml || true
14 changes: 6 additions & 8 deletions docker/pbs/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,8 @@ echo " "
exit_code=0
canary -d run --show-excluded-tests -w -b scheduler=pbs ./examples || exit_code=$?
if [ "${exit_code}" -ne 30 ]; then
cat TestResults/.canary/config || true
cat TestResults/.canary/batches/*/*/resource_pool.json || true
cat TestResults/.canary/batches/*/*/canary-out.txt || true
cat .canary/cache/canary_hpc/batches/*/*/resource_pool.json || true
cat .canary/cache/canary_hpc/batches/*/*/canary-out.txt || true
cat TestResults/basic/second/second/canary-out.txt || true
cat TestResults/basic/second/second/canary-err.txt || true
exit 1
Expand All @@ -68,9 +67,8 @@ echo " "
exit_code=0
canary -d run --show-excluded-tests -w -b scheduler=pbs -b spec=count:3 ./examples || exit_code=$?
if [ "${exit_code}" -ne 30 ]; then
cat TestResults/.canary/config || true
cat TestResults/.canary/batches/*/*/resource_pool.json || true
cat TestResults/.canary/batches/*/*/canary-out.txt || true
cat .canary/cache/canary_hpc/batches/*/*/resource_pool.json || true
cat .canary/cache/canary_hpc/batches/*/*/canary-out.txt || true
cat TestResults/basic/second/second/canary-out.txt || true
cat TestResults/basic/second/second/canary-err.txt || true
exit 1
Expand All @@ -80,5 +78,5 @@ echo " "
echo " "
echo "----------------------- Done! ----------------------"
# Artifacts
canary -C TestResults report junit create -o $CI_PROJECT_DIR/junit.xml || true
canary -C TestResults report cdash create -d $CI_PROJECT_DIR/xml || true
canary report junit create -o $CI_PROJECT_DIR/junit.xml || true
canary report cdash create -d $CI_PROJECT_DIR/xml || true
23 changes: 10 additions & 13 deletions docker/slurm/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,8 @@ echo " "
exit_code=0
canary -d run --show-excluded-tests -w -b scheduler=slurm ./examples || exit_code=$?
if [ "${exit_code}" -ne 30 ]; then
cat TestResults/.canary/config || true
cat TestResults/.canary/batches/*/*/resource_pool.json || true
cat TestResults/.canary/batches/*/*/canary-out.txt || true
cat .canary/cache/canary_hpc/batches/*/*/resource_pool.json || true
cat .canary/cache/canary_hpc/batches/*/*/canary-out.txt || true
cat TestResults/basic/second/second/canary-out.txt || true
cat TestResults/basic/second/second/canary-err.txt || true
exit 1
Expand All @@ -32,9 +31,8 @@ echo " "
exit_code=0
canary -d run --show-excluded-tests -w -b scheduler=slurm -b spec=count:3 ./examples || exit_code=$?
if [ "${exit_code}" -ne 30 ]; then
cat TestResults/.canary/config || true
cat TestResults/.canary/batches/*/*/resource_pool.json || true
cat TestResults/.canary/batches/*/*/canary-out.txt || true
cat .canary/cache/canary_hpc/batches/*/*/resource_pool.json || true
cat .canary/cache/canary_hpc/batches/*/*/canary-out.txt || true
cat TestResults/basic/second/second/canary-out.txt || true
cat TestResults/basic/second/second/canary-err.txt || true
exit 1
Expand All @@ -47,9 +45,8 @@ echo " "
exit_code=0
canary -d run --show-excluded-tests -w -b scheduler=slurm -b spec=count:3,layout:atomic ./examples || exit_code=$?
if [ "${exit_code}" -ne 30 ]; then
cat TestResults/.canary/config || true
cat TestResults/.canary/batches/*/*/resource_pool.json || true
cat TestResults/.canary/batches/*/*/canary-out.txt || true
cat .canary/cache/canary_hpc/batches/*/*/resource_pool.json || true
cat .canary/cache/canary_hpc/batches/*/*/canary-out.txt || true
cat TestResults/basic/second/second/canary-out.txt || true
cat TestResults/basic/second/second/canary-err.txt || true
exit 1
Expand All @@ -62,8 +59,8 @@ echo " "
exit_code=0
canary -d run --show-excluded-tests -w -b scheduler=slurm -b spec=count:auto,layout:flat ./examples || exit_code=$?
if [ "${exit_code}" -ne 30 ]; then
cat TestResults/.canary/batches/*/*/canary-out.txt || true
cat TestResults/.canary/batches/*/*/resource_pool.json || true
cat .canary/cache/canary_hpc/batches/*/*/resource_pool.json || true
cat .canary/cache/canary_hpc/batches/*/*/canary-out.txt || true
cat TestResults/basic/second/second/canary-out.txt || true
cat TestResults/basic/second/second/canary-err.txt || true
exit 1
Expand All @@ -74,5 +71,5 @@ echo " "
echo " "
echo "----------------------- Done! ----------------------"
# Artifacts
canary -C TestResults report junit create -o $CI_PROJECT_DIR/junit.xml || true
canary -C TestResults report cdash create -d $CI_PROJECT_DIR/xml || true
canary report junit create -o $CI_PROJECT_DIR/junit.xml || true
canary report cdash create -d $CI_PROJECT_DIR/xml || true
Loading
Loading