Pack tests - part A

[achamayou]

Fresh start on remaining parts of #7417 to pack the tests more efficiently. A first batch that cuts down ~4 minutes from CI time.

Commit	CI Time
main	28m
6f92e74	26m
ace4790	25m
4ba508d	24m

[achamayou]

Really strange failure, with the schema endpoint returning an empty body:

2025-11-09 15:31:06.005 | ERROR    | {schema} infra.runner:log_exception:194 - Failure in schema: JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
Traceback (most recent call last):

  File "/usr/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
    self.run()
    └ <Thread(schema, started 127166095406784)>
  File "/usr/lib/python3.12/threading.py", line 1012, in run
    self._target(*self._args, **self._kwargs)
    │             │             └ <Thread(schema, started 127166095406784)>
    │             └ <Thread(schema, started 127166095406784)>
    └ <Thread(schema, started 127166095406784)>
  File "/__w/CCF/CCF/tests/schema.py", line 127, in run
    if not fetch_schema(
  File "/__w/CCF/CCF/tests/schema.py", line 39, in fetch_schema
    response_body = api_response.body.json()
                    └ Response(status_code=200, body=b'', seqno=None, view=None, headers=Headers({'content-length': '0', 'content-type': 'application/...
  File "/__w/CCF/CCF/tests/infra/clients.py", line 234, in json
    return self._response.json()
           └ b''
  File "/__w/CCF/CCF/build/env/lib/python3.12/site-packages/httpx/_models.py", line 756, in json
    return jsonlib.loads(self.text, **kwargs)
           │             │            └ {}
           │             └ <Response [200 OK]>
           └ <module 'json' from '/usr/lib/python3.12/json/__init__.py'>
  File "/usr/lib/python3.12/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           │                       └ ''
           └ <json.decoder.JSONDecoder object at 0x73a85990c230>
  File "/usr/lib/python3.12/json/decoder.py", line 338, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               │               │      │  └ ''
               │               │      └ <built-in method match of re.Pattern object at 0x73a85972c2b0>
               │               └ ''
               └ <json.decoder.JSONDecoder object at 0x73a85990c230>
  File "/usr/lib/python3.12/json/decoder.py", line 356, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
          │                                  └ ''
          └ <class 'json.decoder.JSONDecodeError'>

https://github.com/microsoft/CCF/actions/runs/19175402504/job/54911735292?pr=7448

[achamayou]

This is caused by the GOV_API_SCHEMA_2023_06_01_PREVIEW cmake var being expanded in api_schema.h.in. The test failing is comparing the contents of the file to what is returned by /gov/api?api-version=2023-06-01-preview.

In a build with a clean cmake cache, GOV_API_SCHEMA_2023_06_01_PREVIEW is empty, and so the endpoint legitimately returns an empty body.

I am restoring the cmake variable for now, and will PR a separate removal of 2023-06-01-preview, or at least its deprecation for 7.x (depending on the usage situation).

[achamayou]

2024-07-01 is mostly a fairly small extension of 2023-06-01-preview to add /gov/service/js-modules/* endpoints, which is why we stopped duplicating the governance API tests. I think we can deprecate 2023-06-01-preview in 7.x, and drop it in 8.x. There is very little code that will be removed when it goes.

[Copilot]

Pull Request Overview

This PR consolidates Python test files to reduce CI execution time by approximately 4 minutes (from 28m to 24m). The changes pack multiple test suites that were previously run separately into combined test runners using the ConcurrentRunner framework.

Key changes:

• Consolidates election.py, rotation.py, committable.py, and reconfiguration.py tests into nodes.py runner
• Merges membership.py and memberclient.py tests into governance.py runner
• Updates test suite imports from election to nodes module
• Removes standalone test definitions from CMakeLists.txt in favor of consolidated test runners

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/suite/test_suite.py	Updated import from election to nodes module and replaced test function references accordingly
tests/rotation.py	Deleted file - functionality moved to nodes.run_rotations()
tests/reconfiguration.py	Removed __main__ block and ConcurrentRunner import since tests are now called from nodes.py
tests/nodes.py	Added run_rotations() function, imports for reconfiguration and committable, and consolidated multiple test runners into single concurrent runner
tests/membership.py	Removed __main__ block and unused imports (infra.e2e_args, infra.consortium, ConcurrentRunner, memberclient)
tests/infra/runner.py	Changed logging sink from sys.stderr to sys.stdout for better log output handling
tests/governance.py	Added imports for memberclient and membership, added test runners for both modules to consolidate execution
tests/committable.py	Removed __main__ block since test is now called from nodes.py
CMakeLists.txt	Removed individual test definitions (rotation_test, election_test, reconfiguration_test, committable_suffix_test, membership_api_1) and replaced with consolidated nodes_test; moved consistency_trace_validation into LONG_TESTS conditional block