Skip to content

Conversation

@aadityare
Copy link
Collaborator

This PR creates a testing framework to feed a YAML file, run test-cases and capture relevant output. Mainly helps bulk-run ShardGuard with multiple prompts and capture metrics including

  • Time taken per-run, Average run-time (hardware environment specific, can change based on specs)
  • Success Rate, verbose errors and warnings

Can be easily extended to capture further required metrics.

Note: Needs to be merged with Execution Implementation (perhaps locally) to test/work

Sample Output:

📊 TEST SUMMARY
======================================================================

Total Tests: 20
✅ Passed: 1
⚠️  Warnings: 15
❌ Failed: 4

Success Rate: 80.0%
⏱️  Total Time: 1552.75s
⏱️  Average Time: 77.64s per test```

added information about SGTest, a testing framework for ShardGuard, including its purpose and usage
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant