Skip to content

Commit 7662be6

Browse files
feat: Add test count summary to display total and skipped tests (#307)
## Summary Adds a test count summary that displays at the end of each test run showing the total number of tests attempted, how many were skipped, and how many actually ran. ## Changes - **New `CountingTestLogger` wrapper** (`framework/ldtest/test_logger.go`): - Wraps any existing `TestLogger` implementation - Tracks test counts by intercepting `TestStarted` and `TestSkipped` calls - Thread-safe using `sync.Mutex` for concurrent test execution - Provides `GetCounts()` method to retrieve counts - **Updated test execution** (`main.go`): - Wraps the configured test logger with `CountingTestLogger` - Prints summary after test completion: `Test Summary: X total, Y skipped, Z ran` - Summary appears regardless of pass/fail status ## Example Output ``` All tests passed Test Summary: 150 total, 12 skipped, 138 ran ``` ## Important Review Points 1. **No unit tests added**: While existing tests pass and the code compiles, I didn't add specific unit tests for the new `CountingTestLogger`. Consider whether these should be added. 2. **Output placement**: The summary appears after the existing pass/fail messages and any failure listings. Verify this placement meets expectations. 3. **Count interpretation**: - "Total" = all tests that were attempted (called `TestStarted`) - "Skipped" = tests filtered out or explicitly skipped (called `TestSkipped`) - "Ran" = Total - Skipped 4. **Cannot test with real SDK**: I verified the code compiles, passes existing tests, and passes linting, but couldn't test against an actual SDK test service to see the output in practice. --- **Requirements** - [ ] I have added test coverage for new or changed functionality *(No unit tests added for CountingTestLogger)* - [x] I have followed the repository's [pull request submission guidelines](../blob/master/CONTRIBUTING.md#submitting-pull-requests) - [x] I have validated my changes against all supported platform versions *(Code compiles, existing tests pass, linting passes)* **Related issues** Requested by [email protected] in Slack thread: https://launchdarkly.slack.com/archives/C03DEV5CMFD/p1761927792275089 **Describe the solution you've provided** Implemented a transparent wrapper (`CountingTestLogger`) that tracks test counts by intercepting `TestStarted` and `TestSkipped` calls. The wrapper delegates all calls to the underlying logger, ensuring existing functionality remains unchanged. At the end of test execution, a summary line is printed showing total tests, skipped tests, and tests that actually ran. **Describe alternatives you've considered** 1. **Modifying the `Results` struct**: Could have added count fields to `Results`, but this would require more invasive changes throughout the test execution logic. 2. **Only showing total and skipped**: The requirement asked for "total ran" and "skipped", but showing all three numbers (total, skipped, ran) provides more clarity about what "ran" means. 3. **Different output format**: Could format as "X tests ran (Y skipped)" but the current format "X total, Y skipped, Z ran" is more explicit. **Additional context** Link to Devin run: https://app.devin.ai/sessions/61b07c9217f54e62848d32fc3b5136b1 Requested by: [email protected] --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: [email protected] <[email protected]>
1 parent 3d958bc commit 7662be6

File tree

2 files changed

+54
-1
lines changed

2 files changed

+54
-1
lines changed

framework/ldtest/test_logger.go

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import (
44
"fmt"
55
"os"
66
"strings"
7+
"sync"
78

89
"github.com/launchdarkly/sdk-test-harness/v2/framework"
910
"github.com/launchdarkly/sdk-test-harness/v2/framework/helpers"
@@ -44,6 +45,51 @@ type MultiTestLogger struct {
4445
Loggers []TestLogger
4546
}
4647

48+
type CountingTestLogger struct {
49+
wrapped TestLogger
50+
totalTests int
51+
skippedTests int
52+
mu sync.Mutex
53+
}
54+
55+
func NewCountingTestLogger(wrapped TestLogger) *CountingTestLogger {
56+
return &CountingTestLogger{
57+
wrapped: wrapped,
58+
}
59+
}
60+
61+
func (c *CountingTestLogger) TestStarted(id TestID) {
62+
c.mu.Lock()
63+
c.totalTests++
64+
c.mu.Unlock()
65+
c.wrapped.TestStarted(id)
66+
}
67+
68+
func (c *CountingTestLogger) TestError(id TestID, err error) {
69+
c.wrapped.TestError(id, err)
70+
}
71+
72+
func (c *CountingTestLogger) TestFinished(id TestID, result TestResult, debugOutput framework.CapturedOutput) {
73+
c.wrapped.TestFinished(id, result, debugOutput)
74+
}
75+
76+
func (c *CountingTestLogger) TestSkipped(id TestID, reason string) {
77+
c.mu.Lock()
78+
c.skippedTests++
79+
c.mu.Unlock()
80+
c.wrapped.TestSkipped(id, reason)
81+
}
82+
83+
func (c *CountingTestLogger) EndLog(results Results) error {
84+
return c.wrapped.EndLog(results)
85+
}
86+
87+
func (c *CountingTestLogger) GetCounts() (total, skipped int) {
88+
c.mu.Lock()
89+
defer c.mu.Unlock()
90+
return c.totalTests, c.skippedTests
91+
}
92+
4793
func (c ConsoleTestLogger) TestStarted(id TestID) {
4894
fmt.Printf("[%s]\n", id)
4995
}

main.go

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,11 +79,18 @@ func run(params commandParams) (*ldtest.Results, error) {
7979
}}
8080
}
8181

82-
results := sdktests.RunSDKTestSuite(harness, params.filters, testLogger)
82+
countingLogger := ldtest.NewCountingTestLogger(testLogger)
83+
testLogger = countingLogger
84+
85+
results := sdktests.RunSDKTestSuite(harness, params.filters, countingLogger)
8386

8487
fmt.Println()
8588
logErr := testLogger.EndLog(results)
8689

90+
total, skipped := countingLogger.GetCounts()
91+
fmt.Printf("Test Summary: %d total, %d skipped, %d ran\n", total, skipped, total-skipped)
92+
fmt.Println()
93+
8794
if params.stopServiceAtEnd {
8895
fmt.Println("Stopping test service")
8996
if err := harness.StopService(); err != nil {

0 commit comments

Comments
 (0)