Enhance RootPathFilter to support glob-style pattern matching #1125

Copilot · 2025-07-01T22:34:32Z

This PR enhances the existing RootPathFilter configuration to support glob-style pattern matching, helping 1ES PT seamlessly route files for validation when customers download pipeline artifacts.

Changes

Enhanced RootPathFilter

Modified existing RootPathFilter to automatically detect and support glob patterns
Supports common glob patterns:
- * - matches any characters except path separators
- ** - matches any characters including path separators (recursive)
Falls back to legacy path prefix matching when no glob patterns are detected
Maintains full backward compatibility - existing configurations continue to work unchanged

Pattern Matching Implementation

Implemented PathPatternMatcher utility class using .NET's built-in Microsoft.Extensions.FileSystemGlobbing
Cross-platform compatible (handles both Windows and Unix path separators)
Leverages Microsoft's well-tested globbing functionality for reliability

Documentation Updates

Updated help text in ValidationArgs.cs to document glob pattern support
Added comprehensive documentation in IConfiguration.cs comments
Clear indication of both legacy and new pattern matching capabilities

Example Usage

{
  "RootPathFilter": "src/**/*.cs;bin/*.dll;tests/**/test*.json"
}

Or with legacy path prefixes:

{
  "RootPathFilter": "src/;bin/"
}

This allows flexible file matching instead of rigid path prefix matching, enabling more precise control over which files are validated while maintaining full backward compatibility.

Backward Compatibility

All existing RootPathFilter configurations continue to work exactly as before
No breaking changes to the API or configuration format
Glob patterns are detected automatically - no configuration changes needed
Single configuration property handles both use cases seamlessly

Fixes #1040.

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

…n matching logic Co-authored-by: pragnya17 <[email protected]>

Co-authored-by: pragnya17 <[email protected]>

src/Microsoft.Sbom.Api/Filters/DownloadedRootPathFilter.cs

test/Microsoft.Sbom.Api.Tests/Filters/DownloadedRootPathFilterTests.cs

Co-authored-by: pragnya17 <[email protected]>

github-actions · 2025-07-02T18:56:31Z

This PR changes files in the API project. Does it change any of the API interfaces in any way? Please note that this includes the following types of changes:

Changing the signature of an existing interface method
Adding a new method to an existing interface
Adding a required data member to a class that an existing interface method consumes

Because any of these changes can potentially break a downstream consumer with customized interface implementations, these changes need to be treated as breaking changes. Please do one of the following:

Option 1 - Publish this as a breaking change

Update the documentation to show the new functionality
Bump the major version in the next release
Be sure to highlight the breaking changes in the release notes

Option 2 - Refactor the changes to be non-breaking

Review this commit, which adds a new interface in a backward-compatible way
Refactor the change to follow this pattern so that existing interfaces are left completely intact
Bump the minor version in the next release

pragnya17 · 2025-07-02T18:56:32Z

/azp run

pragnya17 · 2025-07-02T20:00:38Z

There are several PathPatternMatcher tests that are failing, can you diagnose and fix them?

…ldcard pattern Co-authored-by: pragnya17 <[email protected]>

Copilot · 2025-07-02T20:14:02Z

There are several PathPatternMatcher tests that are failing, can you diagnose and fix them?

Fixed the PathPatternMatcher tests. The issue was with cross-platform path compatibility and improper handling of the double wildcard (**) pattern. The ** pattern now correctly matches zero or more directory segments, and all path handling is normalized for cross-platform compatibility. All PathPatternMatcher tests are now passing. Commit: b3b21d9

pragnya17 · 2025-07-02T22:23:58Z

/azp run

github-actions · 2025-07-09T16:59:08Z

This PR changes files in the API project. Does it change any of the API interfaces in any way? Please note that this includes the following types of changes:

Changing the signature of an existing interface method
Adding a new method to an existing interface
Adding a required data member to a class that an existing interface method consumes

Because any of these changes can potentially break a downstream consumer with customized interface implementations, these changes need to be treated as breaking changes. Please do one of the following:

Option 1 - Publish this as a breaking change

Update the documentation to show the new functionality
Bump the major version in the next release
Be sure to highlight the breaking changes in the release notes

Option 2 - Refactor the changes to be non-breaking

Review this commit, which adds a new interface in a backward-compatible way
Refactor the change to follow this pattern so that existing interfaces are left completely intact
Bump the minor version in the next release

github-actions · 2025-07-22T18:54:17Z

This PR changes files in the API project. Does it change any of the API interfaces in any way? Please note that this includes the following types of changes:

Changing the signature of an existing interface method
Adding a new method to an existing interface
Adding a required data member to a class that an existing interface method consumes

Because any of these changes can potentially break a downstream consumer with customized interface implementations, these changes need to be treated as breaking changes. Please do one of the following:

Option 1 - Publish this as a breaking change

Update the documentation to show the new functionality
Bump the major version in the next release
Be sure to highlight the breaking changes in the release notes

Option 2 - Refactor the changes to be non-breaking

Review this commit, which adds a new interface in a backward-compatible way
Refactor the change to follow this pattern so that existing interfaces are left completely intact
Bump the minor version in the next release

pragnya17 · 2025-07-22T19:04:23Z

/azp run

pragnya17 · 2025-07-23T18:29:21Z

I actually want to remove adding to the IConfiguration, and just modify the existing RootPathFilter to support the glob-style matching pattern. could you please do that>

github-actions · 2025-07-23T18:30:03Z

This PR changes files in the API project. Does it change any of the API interfaces in any way? Please note that this includes the following types of changes:

Changing the signature of an existing interface method
Adding a new method to an existing interface
Adding a required data member to a class that an existing interface method consumes

Because any of these changes can potentially break a downstream consumer with customized interface implementations, these changes need to be treated as breaking changes. Please do one of the following:

Option 1 - Publish this as a breaking change

Update the documentation to show the new functionality
Bump the major version in the next release
Be sure to highlight the breaking changes in the release notes

Option 2 - Refactor the changes to be non-breaking

Review this commit, which adds a new interface in a backward-compatible way
Refactor the change to follow this pattern so that existing interfaces are left completely intact
Bump the minor version in the next release

…tPathPatterns property Co-authored-by: pragnya17 <[email protected]>

Copilot · 2025-07-23T18:51:54Z

I actually want to remove adding to the IConfiguration, and just modify the existing RootPathFilter to support the glob-style matching pattern. could you please do that>

Done! I've refactored the implementation to remove the separate RootPathPatterns property and instead enhanced the existing RootPathFilter to support glob-style pattern matching automatically.

The RootPathFilter now:

Detects glob patterns (*, **, ?) automatically and switches to pattern matching mode
Falls back to legacy path prefix matching when no glob patterns are detected
Maintains full backward compatibility with existing configurations

All tests pass and the implementation is much cleaner with a single configuration property handling both use cases. Commit: 5fd117a

pragnya17 · 2025-07-23T19:08:56Z

src/Microsoft.Sbom.Api/Filters/DownloadedRootPathFilter.cs

+                if (patterns.Count > 0)
+                {
+                    skipValidation = false;
+                }


re-evaluate if this is needed

Simplified the logic by replacing the if block with a direct assignment: skipValidation = patterns.Count == 0. This achieves the same behavior with cleaner code.

pragnya17 · 2025-07-23T19:10:02Z

src/Microsoft.Sbom.Api/Filters/DownloadedRootPathFilter.cs

+                    }
+                }
+
+                // Only skip validation if we actually have valid paths


re-evaluate if this is necessary

Simplified the logic by replacing the if block with a direct assignment: skipValidation = validPaths.Count == 0. This achieves the same behavior with cleaner code.

pragnya17 · 2025-07-23T19:33:58Z

test/Microsoft.Sbom.Api.Tests/Utils/PathPatternMatcherTests.cs

+    }
+
+    [TestMethod]
+    public void PathPatternMatcher_ComplexPattern_MatchesCorrectly()


can you add a test case here that ensures 0 directories are matched when given **

Added test case PathPatternMatcher_DoubleWildcard_MatchesZeroDirectories that verifies ** pattern correctly matches paths with zero intermediate directories (e.g., src/**/*.txt matches src/file.txt).

…t for ** matching zero directories Co-authored-by: pragnya17 <[email protected]>

github-actions · 2025-07-23T20:44:24Z

This PR changes files in the API project. Does it change any of the API interfaces in any way? Please note that this includes the following types of changes:

Changing the signature of an existing interface method
Adding a new method to an existing interface
Adding a required data member to a class that an existing interface method consumes

Because any of these changes can potentially break a downstream consumer with customized interface implementations, these changes need to be treated as breaking changes. Please do one of the following:

Option 1 - Publish this as a breaking change

Update the documentation to show the new functionality
Bump the major version in the next release
Be sure to highlight the breaking changes in the release notes

Option 2 - Refactor the changes to be non-breaking

Review this commit, which adds a new interface in a backward-compatible way
Refactor the change to follow this pattern so that existing interfaces are left completely intact
Bump the minor version in the next release

github-actions · 2025-07-23T20:44:28Z

This PR changes files in the API project. Does it change any of the API interfaces in any way? Please note that this includes the following types of changes:

Changing the signature of an existing interface method
Adding a new method to an existing interface
Adding a required data member to a class that an existing interface method consumes

Because any of these changes can potentially break a downstream consumer with customized interface implementations, these changes need to be treated as breaking changes. Please do one of the following:

Option 1 - Publish this as a breaking change

Update the documentation to show the new functionality
Bump the major version in the next release
Be sure to highlight the breaking changes in the release notes

Option 2 - Refactor the changes to be non-breaking

Review this commit, which adds a new interface in a backward-compatible way
Refactor the change to follow this pattern so that existing interfaces are left completely intact
Bump the minor version in the next release

DaveTryon · 2025-07-24T22:48:41Z

src/Microsoft.Sbom.Api/Utils/PathPatternMatcher.cs

+/// <summary>
+/// Provides pattern matching functionality for file paths using glob-style patterns.
+/// </summary>
+public static class PathPatternMatcher


.NET already provides cross-platform support for globbing that we should leverage unless there's some reason that it doesn't do what we need it to do. See https://learn.microsoft.com/en-us/dotnet/core/extensions/file-globbing for details and syntax.

Great suggestion! I've refactored the PathPatternMatcher to use .NET's built-in Microsoft.Extensions.FileSystemGlobbing library instead of the custom implementation. This provides better reliability, cross-platform support, and leverages Microsoft's well-tested globbing functionality.

Key changes:

Added Microsoft.Extensions.FileSystemGlobbing dependency

Replaced custom regex-based pattern matching with .NET's Matcher class

Enhanced cross-platform path handling for both Windows and Unix paths

Updated tests to reflect .NET globbing behavior (note: ? wildcard not supported by .NET implementation)

All tests passing with improved maintainability

Commit: 806585d

DaveTryon · 2025-07-24T22:52:51Z

src/Microsoft.Sbom.Common/Config/IConfiguration.cs

    /// Gets or sets if you're downloading only a part of the drop using the '-r' or 'root' parameter
    /// in the drop client, specify the same string value here in order to skip
    /// validating paths that are not downloaded.
+    /// Supports both path prefixes (legacy) and glob-style patterns (* and **) for flexible file path matching.


A comment like this should appear in the help text in ValidationArgs.cs, as well as in the docs. If you change the help text, you can then run the tool and copy/paste directly into the docs. Be careful of your terminal width, since the line breaks will shift and we want to keep them as stable as possible.

@copilot can you address this comment

Updated the help text in ValidationArgs.cs to document the glob-style pattern support for RootPathFilter. The help text now clearly indicates that both path prefixes (legacy) and glob-style patterns (* and **) are supported for flexible file path matching. Commit: ba7f30d

… library Co-authored-by: DaveTryon <[email protected]>

…ionArgs Co-authored-by: pragnya17 <[email protected]>

Initial plan

448f77b

Copilot AI assigned Copilot and pragnya17 Jul 1, 2025

Copilot started work on behalf of pragnya17 July 1, 2025 22:34 View session

Copilot AI and others added 2 commits July 1, 2025 22:51

Add RootPathPatterns configuration and implementation - fixing patter…

bbca699

…n matching logic Co-authored-by: pragnya17 <[email protected]>

Complete RootPathPatterns implementation with working pattern matching

d7f04c3

Co-authored-by: pragnya17 <[email protected]>

Copilot AI changed the title ~~[WIP] Consider changing RootPathFilter to accept pattern matching~~ Add RootPathPatterns configuration for glob-style pattern matching Jul 1, 2025

Copilot AI requested a review from pragnya17 July 1, 2025 22:56

Copilot finished work on behalf of pragnya17 July 1, 2025 22:56

pragnya17 requested changes Jul 2, 2025

View reviewed changes

src/Microsoft.Sbom.Api/Filters/DownloadedRootPathFilter.cs Outdated Show resolved Hide resolved

test/Microsoft.Sbom.Api.Tests/Filters/DownloadedRootPathFilterTests.cs Outdated Show resolved Hide resolved

Copilot started work on behalf of pragnya17 July 2, 2025 18:19 View session

Address PR feedback: restructure code flow and fix test comment

980d3e8

Co-authored-by: pragnya17 <[email protected]>

Copilot AI requested a review from pragnya17 July 2, 2025 18:35

Copilot finished work on behalf of pragnya17 July 2, 2025 18:35

Copilot started work on behalf of pragnya17 July 2, 2025 20:00 View session

Fix PathPatternMatcher for cross-platform compatibility and double wi…

b3b21d9

…ldcard pattern Co-authored-by: pragnya17 <[email protected]>

Copilot finished work on behalf of pragnya17 July 2, 2025 20:14

Merge branch 'main' into copilot/fix-1040

e4896c8

Merge branch 'main' into copilot/fix-1040

d6b3d56

Copilot started work on behalf of pragnya17 July 23, 2025 18:29 View session

Merge branch 'main' into copilot/fix-1040

72397ff

Refactor RootPathFilter to support glob patterns without separate Roo…

5fd117a

…tPathPatterns property Co-authored-by: pragnya17 <[email protected]>

Copilot finished work on behalf of pragnya17 July 23, 2025 18:52

pragnya17 requested changes Jul 23, 2025

View reviewed changes

Copilot started work on behalf of pragnya17 July 23, 2025 19:34 View session

Address PR review feedback: simplify skipValidation logic and add tes…

2d337df

…t for ** matching zero directories Co-authored-by: pragnya17 <[email protected]>

Copilot AI requested a review from pragnya17 July 23, 2025 19:45

Copilot finished work on behalf of pragnya17 July 23, 2025 19:45

Merge branch 'main' into copilot/fix-1040

8025588

pragnya17 approved these changes Jul 23, 2025

View reviewed changes

DaveTryon reviewed Jul 24, 2025

View reviewed changes

Copilot started work on behalf of DaveTryon July 24, 2025 22:48 View session

DaveTryon reviewed Jul 24, 2025

View reviewed changes

Refactor PathPatternMatcher to use .NET's built-in FileSystemGlobbing…

806585d

… library Co-authored-by: DaveTryon <[email protected]>

Copilot AI requested a review from DaveTryon July 24, 2025 23:13

Copilot finished work on behalf of DaveTryon July 24, 2025 23:13

Copilot started work on behalf of pragnya17 August 7, 2025 21:34 View session

Add glob pattern documentation to RootPathFilter help text in Validat…

ba7f30d

…ionArgs Co-authored-by: pragnya17 <[email protected]>

Copilot AI changed the title ~~Add RootPathPatterns configuration for glob-style pattern matching~~ Enhance RootPathFilter to support glob-style pattern matching Aug 7, 2025

Copilot finished work on behalf of pragnya17 August 7, 2025 21:42

Copilot AI requested a review from pragnya17 August 7, 2025 21:42

Enhance RootPathFilter to support glob-style pattern matching #1125

Are you sure you want to change the base?

Enhance RootPathFilter to support glob-style pattern matching #1125

Uh oh!

Conversation

Copilot AI commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Enhanced RootPathFilter

Pattern Matching Implementation

Documentation Updates

Example Usage

Backward Compatibility

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jul 2, 2025

Option 1 - Publish this as a breaking change

Option 2 - Refactor the changes to be non-breaking

Uh oh!

pragnya17 commented Jul 2, 2025

Uh oh!

pragnya17 commented Jul 2, 2025

Uh oh!

Copilot AI commented Jul 2, 2025

Uh oh!

pragnya17 commented Jul 2, 2025

Uh oh!

github-actions bot commented Jul 9, 2025

Option 1 - Publish this as a breaking change

Option 2 - Refactor the changes to be non-breaking

Uh oh!

github-actions bot commented Jul 22, 2025

Option 1 - Publish this as a breaking change

Option 2 - Refactor the changes to be non-breaking

Uh oh!

pragnya17 commented Jul 22, 2025

Uh oh!

pragnya17 commented Jul 23, 2025

Uh oh!

github-actions bot commented Jul 23, 2025

Option 1 - Publish this as a breaking change

Option 2 - Refactor the changes to be non-breaking

Uh oh!

Copilot AI commented Jul 23, 2025

Uh oh!

pragnya17 Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

pragnya17 Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

pragnya17 Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 23, 2025

Option 1 - Publish this as a breaking change

Option 2 - Refactor the changes to be non-breaking

Uh oh!

github-actions bot commented Jul 23, 2025

Option 1 - Publish this as a breaking change

Option 2 - Refactor the changes to be non-breaking

Uh oh!

DaveTryon Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

DaveTryon Jul 24, 2025

Choose a reason for hiding this comment

Copilot AI commented Jul 1, 2025 •

edited

Loading