[go: up one dir, main page]

Skip to content

Draft: PoC: Implement matrix expressions

What does this MR do and why?

This MR is a Proof of Concept for implementing matrix expressions in needs:parallel:matrix values

Issue: Support dynamic job dependencies in `needs:para... (#423553)

Specific proposal being tested by this PoC: #423553 (comment 2601916794)

Demonstration

I used this config to test a mildly complex matrix example in my GDK

variables:
  BASE_IMAGE: "alpine:3.18"

.matrix_config: &matrix_config
  parallel:
    matrix:
      - OS: ["ubuntu", "alpine", "centos"] 
        ARCH: ["amd64", "arm64"]
        ENV: ["dev", "prod"]

stages:
  - build
  - test
  - package

build:
  stage: build
  image: $BASE_IMAGE
  script:
    - echo "Building $OS-$ARCH for $ENV environment"
  <<: *matrix_config

unit-test:
  stage: test
  image: ubuntu:22.04
  script:
    - echo "Running unit tests for $OS-$ARCH-$ENV"
  needs:
    - job: build
      parallel:
        matrix:
          - OS: $[[ matrix.OS ]]
            ARCH: $[[ matrix.ARCH ]] 
            ENV: $[[ matrix.ENV ]]
  <<: *matrix_config

integration-test: # only depend on the ENV=prod jobs
  stage: test
  image: ubuntu:22.04
  script:
    - echo "Running integration tests for $OS-$ARCH"
  needs:
    - job: build
      parallel:
        matrix:
          - OS: $[[ matrix.OS ]]
            ARCH: $[[ matrix.ARCH ]]
            ENV: "prod"
  parallel:
    matrix:
      - OS: ["ubuntu", "alpine"]
        ARCH: ["amd64", "arm64"]
        ENV: ["prod"]

package: # only depend on the ENV=prod jobs
  stage: package
  image: $BASE_IMAGE
  script:
    - echo "Packaging $OS-$ARCH-$ENV"
  needs:
    - job: unit-test
      parallel:
        matrix:
          - OS: $[[ matrix.OS ]]
            ARCH: $[[ matrix.ARCH ]]
            ENV: $[[ matrix.ENV ]]
    - job: integration-test
      parallel:
        matrix:
          - OS: $[[ matrix.OS ]]
            ARCH: $[[ matrix.ARCH ]]
            ENV: $[[ matrix.ENV ]]
  parallel:
    matrix:
      - OS: ["ubuntu", "alpine"] 
        ARCH: ["amd64", "arm64"]
        ENV: ["prod"]

It created a pipeline with the expected jobs

Screenshot 2025-08-27 at 17.57.45.png

It's challenging to see the needs relation of matrix jobs in the pipeline graph, so let's look at each of the job groups individually

integration-test

Screenshot 2025-08-27 at 17.57.53.png

unit-test

Screenshot 2025-08-27 at 17.58.27.png

package

Screenshot 2025-08-27 at 17.58.21.png

Edited by Avielle Wolfe

Merge request reports

Loading