Draft: PoC: Implement matrix expressions
What does this MR do and why?
This MR is a Proof of Concept for implementing matrix expressions in needs:parallel:matrix
values
Issue: Support dynamic job dependencies in `needs:para... (#423553)
Specific proposal being tested by this PoC: #423553 (comment 2601916794)
Demonstration
I used this config to test a mildly complex matrix example in my GDK
variables:
BASE_IMAGE: "alpine:3.18"
.matrix_config: &matrix_config
parallel:
matrix:
- OS: ["ubuntu", "alpine", "centos"]
ARCH: ["amd64", "arm64"]
ENV: ["dev", "prod"]
stages:
- build
- test
- package
build:
stage: build
image: $BASE_IMAGE
script:
- echo "Building $OS-$ARCH for $ENV environment"
<<: *matrix_config
unit-test:
stage: test
image: ubuntu:22.04
script:
- echo "Running unit tests for $OS-$ARCH-$ENV"
needs:
- job: build
parallel:
matrix:
- OS: $[[ matrix.OS ]]
ARCH: $[[ matrix.ARCH ]]
ENV: $[[ matrix.ENV ]]
<<: *matrix_config
integration-test: # only depend on the ENV=prod jobs
stage: test
image: ubuntu:22.04
script:
- echo "Running integration tests for $OS-$ARCH"
needs:
- job: build
parallel:
matrix:
- OS: $[[ matrix.OS ]]
ARCH: $[[ matrix.ARCH ]]
ENV: "prod"
parallel:
matrix:
- OS: ["ubuntu", "alpine"]
ARCH: ["amd64", "arm64"]
ENV: ["prod"]
package: # only depend on the ENV=prod jobs
stage: package
image: $BASE_IMAGE
script:
- echo "Packaging $OS-$ARCH-$ENV"
needs:
- job: unit-test
parallel:
matrix:
- OS: $[[ matrix.OS ]]
ARCH: $[[ matrix.ARCH ]]
ENV: $[[ matrix.ENV ]]
- job: integration-test
parallel:
matrix:
- OS: $[[ matrix.OS ]]
ARCH: $[[ matrix.ARCH ]]
ENV: $[[ matrix.ENV ]]
parallel:
matrix:
- OS: ["ubuntu", "alpine"]
ARCH: ["amd64", "arm64"]
ENV: ["prod"]
It created a pipeline with the expected jobs
It's challenging to see the needs
relation of matrix jobs in the pipeline graph, so let's look at each of the job groups individually
integration-test
unit-test
package
Edited by Avielle Wolfe