test-fbc: add integration tests pipeline

[wainersm]

- Description of the problem which is fixed/What is the use case

We want to run the downstream QE tests on new builds of osc-test-catalog image.

- What I did

This is a tekton pipeline that should be triggered by Konflux as integration tests after a osc-test-catalog image is successful built. It will trigger the existing downstream Prow jobs that will run the OSC e2e tests on the new test catalog.

Merging that PR will not enable the triggering as for that we need to configure on Konflux side.

- How to verify it

That's how I verified this works:

• pushed the pipeline definition to https://github.com/wainersm/sandboxed-containers-operator/tree/osc-test-fbc-integration
• At Konflux side, added a new integration tests (named osc-test-catalog-intergration) for the osc-test-catalog application, pointing to the above definition. Configured to trigger on pull request
• Still at Konflux, created a secret named gangway-token with my personal Prow API token
• Opened a fake PR on this repository just to trigger the pipeline on Konflux: [DO NOT MERGE] Update Dockerfile #1228

As you can see in https://github.com/openshift/sandboxed-containers-operator/pull/1228/checks?check_run_id=53619686100 it triggered the pipeline on Konflux that in turn triggered 5 jobs in Prow (each of the tasks named "test-kata" correspond to a Prow job).

The pipeline failed because each task is given 2h to finish but our Prow jobs takes more time than that. It seems that the pipeline can be configured with a different timeout, I tried but it still waiting for the 2h....this is something I will need to figure out.

The triggered prow jobs all failed too because tests failed. It is something that we should be fixing on Prow and/or OSC e2e tests.

- Description for the changelog

N/A

[littlejawa]

Hey @wainersm - This is great!
Looks like the triggering works as expected, and I can see the catalog reference is retrieved properly. This is a huge step forward.

We have a "tests/" folder where I put the "make-test" pipeline, maybe this one should go there too? Just to separate build and test pipelines.
The "test-kata" name comes from the prow job? Any way to display the actual test that was triggered? This is not a big deal - we can see it when we click on it.

Great job anyway - Thanks for looking into this!

[wainersm]

Hey @wainersm - This is great! Looks like the triggering works as expected, and I can see the catalog reference is retrieved properly. This is a huge step forward.

\o/

We have a "tests/" folder where I put the "make-test" pipeline, maybe this one should go there too? Just to separate build and test pipelines. The "test-kata" name comes from the prow job? Any way to display the actual test that was triggered? This is not a big deal - we can see it when we click on it.

I will move the pipeline to the "tests" folder.

yeah, I was also thinking about the "test-kata" name, it would be good if we could display the configuration of each one... this is my first tekton pipeline so I am still learning. But I will check if there is way.

Great job anyway - Thanks for looking into this!

Thanks @littlejawa !

[wainersm]

@littlejawa updated it with:

• now the task (prow-job) has a timeout of 4h. I still have the annotations with timeouts in the Konflux configuration, I still don't know if the annotation in the pipeline is required, what I know is that it doesn't hurt and works :)
• moved the pipeline definition to "tests" folder
• the task is renamed from "test-kata" to "prow-job". In the results table in Github it will have "prow-job" entries and we cannot see what prow job it refers to, you need to open the konflux dashboard. The only way I found to print the specific job name is to create one prow-job task per prow job (instead of using matrix) but then we will be duplicating code....

[openshift-ci]

@wainersm: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[tbuskey]

/lgtm