[DPE-8315] Split integration and release tests

[sinclert-canonical]

This PR proposes the split of integration and release tests into different directories. Splitting tests into different folders (even within the same tests/integration parent one), allow us to define separated spread suites, in order to execute the suites on different GHA events:

• Integration tests will be executed upon pull_request and schedule events.
• Release tests will be executed upon push event targeting the main branch.

Additional changes

• Moved the upgrade_stable.py test to tests/integration/release folder, making it test every stable revision to date.
• Removed the ci.yaml workflow workflow_call trigger. Its only consumer will call integration_tests.yaml instead.
• Extended the release.yaml workflow list of ignored path, to account for recently added workflows.

[sinclert-canonical]

Apparently, all previous stable revisions can be upgraded to latest edge cc @paulomach @carlcsaposs-canonical

[paulomach]

Solid. One thing I wish to change is the spread tasks upgrade test names. Any good reason to use date instead of the revision, other that we have more than one revision per date? I would rather stick with the amd64 revision number as it will be there for all.

[sinclert-canonical]

Any good reason to use date instead of the revision, other that we have more than one revision per date? I would rather stick with the amd64 revision number as it will be there for all.

No other reason than the one you described: there are multiple revisions per every stable (one per CPU arch). The date naming was adopted after Carl's feedback of reducing the number of task.yaml files to maintain, by grouping ~ similar revisions together.

Naming is completely arbitrary and it can be changed to whatever we want. Do you rather do AMD revision naming? Keep in mind that the AMD revision is not always the first one to be promoted to stable (sometimes ARM is first).

[carlcsaposs-canonical]

since we're running integration_test.yaml on PR and on release, can we keep it in ci.yaml and call ci.yaml in release.yaml to avoid duplication (and eliminate the possibility of debugging differences between the two)?

[sinclert-canonical]

I think having a strict separation between workflows that are reusable (i.e. trigger-able upon workflow_call) and workflows that are not will help us identify which ones could be, eventually, promoted to some form of shared workflow setup.

[paulomach]

Anyway will do, let's not block over details.

[carlcsaposs-canonical]

duplicating ci.yaml into 3 separate workflows for the pull_request, push, and schedule events is, imo, an antipattern and one that we've intentionally moved away from in the past

it adds unnecessary duplication & complexity and leads to issues like nightly tests not being consistent with pr tests

I don't understand how that change is necessary/relevant to this pr—the existing architecture can be preserved in a straightforward way

[sinclert-canonical]

Duplicating ci.yaml into 3 separate workflows for the pull_request, push, and schedule events is, imo, an antipattern and one that we've intentionally moved away from in the past

Workflows are not duplicated, because each of them target a different subset of tests / validations:

• PR workflow: linting + unit tests + alert tests + integration tests.
• Release workflow: integration tests + release tests + release lib + release charm.
• Schedule workflow: integration tests + release tests.

As you can see, each workflow has the specific set of steps that make sense for them, while avoiding duplication by extracting those steps that are shared across workflows (i.e. integration tests and release_test) to their own reusable ones.

If you are un-happy with the code duplication between both reusable workflows: trust me, I am too. I suggested you to move the shared Python code to a couple of Python scripts in order to avoid having everything in line, and you refused arguing maintainability concerns (see comment).

[carlcsaposs-canonical]

Naming is completely arbitrary and it can be changed to whatever we want. Do you rather do AMD revision naming? Keep in mind that the AMD revision is not always the first one to be promoted to stable (sometimes ARM is first).

if we're going to use revision naming instead of dates, I would pick the alphabetically first revision to match the pattern used in our github release tags https://github.com/canonical/data-platform-workflows/blob/fa480d077b4e5485efcae80d114ff37414b0a0f3/_cli/data_platform_workflows_cli/craft_tools/promote_legacy.py#L245-L248

but imo date is fine

[paulomach]

Naming is completely arbitrary and it can be changed to whatever we want. Do you rather do AMD revision naming? Keep in mind that the AMD revision is not always the first one to be promoted to stable (sometimes ARM is first).

if we're going to use revision naming instead of dates, I would pick the alphabetically first revision to match the pattern used in our github release tags https://github.com/canonical/data-platform-workflows/blob/fa480d077b4e5485efcae80d114ff37414b0a0f3/_cli/data_platform_workflows_cli/craft_tools/promote_legacy.py#L245-L248

but imo date is fine

ok, we stick with dates

[sinclert-canonical]

PR branch rebased from main branch.

[astrojuanlu]

Aside from @carlcsaposs-canonical 's comment on the release tests not being included in the Allure report, this LGTM