Skip to content

Conversation

@ben-c-at-moz
Copy link
Collaborator

Relevant Links

Bugzilla

Description of Code / Doc Changes

  • Create testkey objects and methods:
    • Add a test to a manifest
    • Automatically balance functional splits
    • Gather tests from a split
    • Convert filenames to testkey dicts
    • Convert testkey dicts to filenames
  • Change chooser to use TestKey
  • Change CI to allow for new logic
  • Change devsetup / git hooks to require addtests
  • Update docs

Process Changes Required

Mark the relevant boxes:

  • Changes CI flow
  • Changes scheduled Beta or DevEdition
  • Changes Git hooks or Github settings

Screenshots or Explanations

Example incident run

Comments or Future Work

  • Addtests method doesn't really support subtests yet

Workflow Checklist

  • Please request reviewers
  • If this is an unblocker, please post in Slack.
  • If asked to address comments, please resolve conversations.
  • If asked to change code, please re-request review from the person who wanted changes.

Thank you!

ben-c-at-moz and others added 8 commits December 1, 2025 16:38
* read manifests in chooser script

* update CI beta flows to take manifests

* typo

* read manifests in chooser script

* update CI beta flows to take manifests

* typo

* attempt to correctly choose tests for win / linux

* switch conftest version gathering from fixture

* debug manifests not respected in mac/lin ci

* debug manifests not respected in mac/lin ci

* debug manifests not respected in mac/lin ci

* debug manifests not respected in mac/lin ci

* mark certain incident tests unstable

* incident manifest

* remove unnecessary incident test

* add manifest maker, change chooser to manifests / git logic only

* remove unstable mark and rely on manifests

* single source of pass/fail truth

* update manifest key; chooser defaults out to a manifest

* lint

* all -> smoke as default manifest

* update manifest key; return win to explicit gecko

* update manifest key; return win to explicit gecko

* update manifest key; return win to explicit gecko

* update manifest key; return win to explicit gecko

* update manifest key; return win to explicit gecko

* README

* README

* mark test unstable
* modify workflows to allow non-smoke test sets

* missed some name changes

* missed some name changes

* missed some name changes

* missed some name changes

* missed some name changes

* missed some name changes

* missed some name changes

* pwsh

* debug win flow

* debug win flow

* add manifests for all and functional

* add functional flow

* filter chosen tests by key.yaml

* fix missing extensions
* revert to using old version, but check that it's valid

* revert to using old version, but check that it's valid

* fix tc for manifests

* other cleanup
Copy link
Collaborator

@Tracy-Walker Tracy-Walker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's go.

@ben-c-at-moz ben-c-at-moz merged commit 92390b4 into main Dec 2, 2025
15 checks passed
@ben-c-at-moz ben-c-at-moz deleted the ben/merge-yamls branch December 2, 2025 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants