diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index c598911f5134e0a3ba90c4ab836e119760b35120..3701bbcd0dde9534cf16d777fe0f125aede8030b 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -182,3 +182,13 @@ To control access to protected variables and runners: - Expand **Variables** - Under **Access protected resources in merge request pipelines**, select or clear the **Allow merge request pipelines to access protected variables and runners** option. + +## Troubleshooting + +### Stop a pipeline early for Git or policy problems + +If a merge request has Git or policy problems, the merge cannot happen, but CI/CD +pipelines are still created. To preserve resources and prevent a long pipeline that can never succeed, +consider creating jobs that run at the beginning of your CI/CD pipeline, as described in +[Tutorial: Stop long merge pipelines early with fail-fast CI/CD jobs](../../tutorials/merge_requests/fail_fast.md). +Configured properly, these jobs fail quickly for common problems. diff --git a/doc/tutorials/merge_requests/fail_fast.md b/doc/tutorials/merge_requests/fail_fast.md new file mode 100644 index 0000000000000000000000000000000000000000..d83ed5c0a5967c45e00b164775324175e41e75f5 --- /dev/null +++ b/doc/tutorials/merge_requests/fail_fast.md @@ -0,0 +1,344 @@ +--- +stage: Create +group: Code Review +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments +description: How the GitLab UI helps you track merge requests from creation to merging. +title: 'Tutorial: Stop long merge pipelines early with fail-fast CI/CD jobs' +--- + +{{< details >}} + +- Tier: Premium, Ultimate +- Offering: GitLab.com, GitLab Self-Managed, GitLab Dedicated + +{{< /details >}} + +You don't want to waste CI/CD minutes on a long pipeline that has no chance of ever succeeding. +When you create fail-fast checks early in your pipeline, developers don't have to wait for hours to +learn about common pipeline failures. + +## Understand the types of failures + +Merge request pipelines can fail for many reasons. Two types of failures are common, and can be +detected quickly: + +- Structural problems in Git, such as merge conflicts or needed rebases, which prevent merging. +- Policy problems, which prevent merging until the project's merge policies are satisfied. + +The example script on this page creates two parallel jobs in the `.pre` stage that runs first in your +pipeline. If either check fails, the entire pipeline fails early. The checks combine the raw speed +and accuracy of a Git dry-run, plus the intelligent, context-aware feedback from the API. + +## Add jobs to your `.gitlab-ci.yml` + +Edit the jobs in this example `.gitlab-ci.yml` to meet your project's needs. The example provides +two jobs: + +- The `check-merge-conflict-git` check focuses only on Git conflicts. It uses a lightweight `alpine:latest` + image, installs Git, and performs a `git merge --no-commit`. If any code conflicts exist, the job + fails. +- The broader `check-merge-status` check uses the merge request API to check the value of `detailed_merge_status`, + and provides human-readable error messages. The job fails on Git problems (like merge conflicts) or policy + issues (like lack of approvals, or unresolved discussions). When testing merge requests in draft status + or with incomplete CI/CD pipelines, this check also considers the underlying value of `can_be_merged`. + Work in progress is not forced out of draft status to satisfy this check. + +The merge request API documentation includes a full list of +[possible values for `detailed_merge_status`](../../api/merge_requests.md#merge-status). + +### Example code for `.gitlab-ci.yml` + +```yaml +#.gitlab-ci.yml + +# ========================================================================== +# 1. WORKFLOW CONTROL +# ========================================================================== +# This section defines the global rules for pipeline creation. It's the +# first thing GitLab evaluates. Its purpose is to ensure pipelines run ONLY +# for the events we care about, preventing wasteful duplicate pipelines. +# +# The logic: +# - Rule 1: Always create a pipeline for merge request events. +# - Rule 2: Always create a pipeline for commits to the default branch (like 'main'). +# - Rule 3: Always create a pipeline for tags (for releases). +# - Rule 4: NEVER create a branch pipeline if a merge request is already open for that branch. +# This rule is the key to prevent duplicate pipelines. +# - Rule 5: A fallback to allow manual pipeline runs from the UI ('web') and for scheduled pipelines. +# +# For more details, see: https://docs.gitlab.com/ee/ci/yaml/workflow.html + +#.gitlab-ci.yml +workflow: + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + - if: $CI_COMMIT_TAG + - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS + when: never + - if: $CI_PIPELINE_SOURCE == 'web' + - if: $CI_PIPELINE_SOURCE == 'schedule' + +# ========================================================================== +# 2. STAGES +# ========================================================================== +stages: + - ❄️ validate + - 🛠 build + - 🚂 test + - ⚙️ Run + - 🚀 staging + - 🐍 deploy + - 🛑 Stop + - 📚 cleanup + +# ========================================================================== +# 3. JOB TEMPLATES & DEFAULTS +# ========================================================================== +.default_rules: + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + +# ========================================================================== +# 4. VALIDATION JOBS +# ========================================================================== +# Job 1: Polls the GitLab API for the detailed merge status. +check-merge-status: + stage: .pre + image: alpine:latest + extends: .default_rules + before_script: + - apk add --no-cache curl jq diffutils + script: + - | + set -euo pipefail + echo "🔍 [API Check] Checking merge request !${CI_MERGE_REQUEST_IID} for mergeability..." + + # --- Configuration --- + POLL_INTERVAL=10 + MAX_ATTEMPTS=18 # 18 attempts * 10s = 3 minutes timeout + API_URL="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${CI_MERGE_REQUEST_IID}" + + # --- Initial API Call --- +# ========================================================================== +# 1. WORKFLOW CONTROL +# ========================================================================== +# This section defines the global rules for pipeline creation. It's the +# first thing GitLab evaluates. Its purpose is to ensure pipelines run ONLY +# for the events we care about, preventing wasteful duplicate pipelines. +# +# The logic is as follows: +# - Rule 1: Always create a pipeline for merge request events. +# - Rule 2: Always create a pipeline for commits to the default branch (e.g., 'main' or 'master'). +# - Rule 3: Always create a pipeline for tags (for releases). +# - Rule 4: NEVER create a branch pipeline if a merge request is already open for that branch. +# This is the key rule to prevent duplicate pipelines. +# - Rule 5: A fallback to allow manual pipeline runs from the UI ('web') and for scheduled pipelines. +# +# For more details, see: https://docs.gitlab.com/ee/ci/yaml/workflow.html +#.gitlab-ci.yml +workflow: + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + - if: $CI_COMMIT_TAG + - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS + when: never + - if: $CI_PIPELINE_SOURCE == 'web' + - if: $CI_PIPELINE_SOURCE == 'schedule' + +# ========================================================================== +# 2. STAGES +# ========================================================================== +stages: + - 🚂 test + +# ========================================================================== +# 3. VALIDATION JOBS +# ========================================================================== +# Job 1: Polls the GitLab API for a detailed merge status. +check-merge-status: + stage: .pre + image: alpine:latest + variables: + # We do not want to clone the repo and use only the MR Api + GIT_STRATEGY: none + before_script: + - apk add --no-cache curl jq diffutils + script: + - | + set -euo pipefail + echo "🔍 [API Check] Checking merge request !${CI_MERGE_REQUEST_IID} for mergeability..." + + # --- Configuration --- + POLL_INTERVAL=10 + TIMEOUT=180 + MAX_ATTEMPTS=$(( ${TIMEOUT} / ${POLL_INTERVAL} )) + API_URL="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${CI_MERGE_REQUEST_IID}" + + # --- Initial API Call --- + echo "Fetching initial MR data and triggering a status recheck..." + PREVIOUS_RESPONSE=$(curl --silent --show-error --fail \ + -H "JOB-TOKEN: ${CI_JOB_TOKEN}" "${API_URL}?with_merge_status_recheck=true" | jq --sort-keys .) + + echo "------------------- Initial Merge Request State -------------------" + echo "${PREVIOUS_RESPONSE}" + echo "---------------------------------------------------------------------" + + # --- Polling Loop --- + for i in $(seq 1 $MAX_ATTEMPTS); do + if [ $i -gt 1 ]; then + sleep ${POLL_INTERVAL} + echo -e "\nAttempt ${i}/${MAX_ATTEMPTS}: Re-querying MR status..." + CURRENT_RESPONSE=$(curl --silent --show-error --fail \ + -H "JOB-TOKEN: ${CI_JOB_TOKEN}" "${API_URL}" | jq --sort-keys .) + + echo "🔄 Diffs from previous state:" + diff -u <(echo "${PREVIOUS_RESPONSE}") <(echo "${CURRENT_RESPONSE}") || true + + PREVIOUS_RESPONSE="${CURRENT_RESPONSE}" + else + echo -e "\nAttempt 1/${MAX_ATTEMPTS}: Analyzing initial MR status..." + CURRENT_RESPONSE="${PREVIOUS_RESPONSE}" + fi + + DETAILED_STATUS=$(echo "${CURRENT_RESPONSE}" | jq -r '.detailed_merge_status') + echo "🔖 Current detailed_merge_status: ${DETAILED_STATUS}" + + case "${DETAILED_STATUS}" in + "mergeable") + echo "✅ [API Check] Success: MR is mergeable. Pipeline can proceed." + exit 0 + ;; + "conflict" | "need_rebase") + echo "❌ [API Check] Fatal Error: MR has a hard conflict. To fix, please '${DETAILED_STATUS}' the source branch." + exit 1 + ;; + "discussions_not_resolved" | "not_approved" | "requested_changes" | "merge_request_blocked" | "not_open" | "security_policy_violations" | "jira_association_missing" | "locked_paths" | "locked_lfs_files" | "title_regex" | "commits_status") + echo "❌ [API Check] Policy Error: MR is blocked by a project policy: '${DETAILED_STATUS}'. Please resolve the issue in the MR view." + exit 1 + ;; + "draft_status" | "ci_still_running") + SIMPLE_STATUS=$(echo "${CURRENT_RESPONSE}" | jq -r '.merge_status') + if [ "${SIMPLE_STATUS}" == "can_be_merged" ]; then + echo "✅ [API Check] Success: Status is '${DETAILED_STATUS}', but MR is otherwise mergeable. Continuing." + exit 0 + else + echo "⏳ [API Check] Status is '${DETAILED_STATUS}' and not yet mergeable. Continuing to poll..." + fi + ;; + *) # All other statuses are asynchronous/transitional + echo "⏳ [API Check] Status is '${DETAILED_STATUS}'. This is a transitional state. Waiting for next poll..." + ;; + esac + done + + echo "⏰ [API Check] Error: Timed out waiting for a terminal merge status." + exit 1 + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' + +# Job 2: Uses Git directly to check for merge conflicts. +check-merge-conflict-git: + stage: .pre + image: alpine:latest + before_script: + - apk add --no-cache git + script: + - | + set -euo pipefail + echo "🔍 [Git Check] Performing a local merge test..." + + git config --global user.name "GitLab CI" + git config --global user.email "gitlab-ci@example.com" + + echo "Fetching target branch '${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}'..." + git fetch origin "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME" + + echo "Attempting to merge target into source branch locally..." + git merge --no-commit --no-ff "origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME" + + echo "✅ [Git Check] Success: No git conflicts detected. The branches can be merged." + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' + +# Example subsequent job that will not run if the above job fails +run-tests: + stage: 🚂 test + script: + - | + echo "--- 🔍 Inspecting CI/CD Variables ---" + echo "Project ID: $CI_PROJECT_ID" + echo "Merge Request IID: $CI_MERGE_REQUEST_IID" + echo "Merge Request Title: '$CI_MERGE_REQUEST_TITLE'" + echo "Source Branch: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME" + echo "Target Branch: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME" + echo "GitLab API v4 URL: $CI_API_V4_URL" + echo "CI Job Token: Is a secret" + echo "------------------------------------" +``` + +### Failure examples from these jobs + +The script creates two jobs. If one or both fail, you might see this on your merge request's +**Pipelines** page: + +![A pipeline that has failed early because of two job failures.](img/script_failure_v18_3.png) + +Failures in the `check-merge-conflict-git` job can look like this: + +```plaintext +$ set -euo pipefail # collapsed multi-line command +🔍 [Git Check] Performing a local merge test... +Fetching target branch 'main'... +From https://gitlab.com/myproject + * branch main -> FETCH_HEAD + * [new branch] main -> origin/main +Attempting to merge target into source branch locally... +Auto-merging content/example.rb +CONFLICT (content): Merge conflict in content/example.rb +Automatic merge failed; fix conflicts and then commit the result. +Cleaning up project directory and file based variables +00:00 +ERROR: Job failed: exit code 1 +``` + +Failures in the `check-merge-status` job can look like this: + +```plaintext +--------------------------------------------------------------------- +Attempt 1/18: Analyzing initial MR status... +🔖 Current detailed_merge_status: approvals_syncing +⏳ [API Check] Status is 'approvals_syncing'. This is a transitional state. Waiting for next poll... +Attempt 2/18: Re-querying merge request status... +🔄 Diffs from previous state: +--- /dev/fd/64 2025-07-27 12:57:20.457034196 +0000 ++++ /dev/fd/65 2025-07-27 12:57:20.458034185 +0000 +@@ -38,7 +38,7 @@ + "closed_by": null, + "created_at": "2025-07-27T12:32:05.613Z", + "description": "Experiment merge request\n\nCloses #2", +- "detailed_merge_status": "approvals_syncing", ++ "detailed_merge_status": "draft_status", + "diff_refs": { + "base_sha": "00000000", + "head_sha": "11111111", +🔖 Current detailed_merge_status: draft_status +⏳ [API Check] Status is 'draft_status' and not yet mergeable. Continuing to poll... +Attempt 3/18: Re-querying merge request status... +🔄 Diffs from previous state: +🔖 Current detailed_merge_status: draft_status +⏳ [API Check] Status is 'draft_status' and not yet mergeable. Continuing to poll... + +[...] + +Attempt 18/18: Re-querying merge request status... +🔄 Diffs from previous state: +🔖 Current detailed_merge_status: draft_status +⏳ [API Check] Status is 'draft_status' and not yet mergeable. Continuing to poll... +⏰ [API Check] Error: Timed out waiting for a terminal merge status. +Cleaning up project directory and file based variables +00:01 +ERROR: Job failed: exit code 1 +``` diff --git a/doc/tutorials/merge_requests/img/script_failure_v18_3.png b/doc/tutorials/merge_requests/img/script_failure_v18_3.png new file mode 100644 index 0000000000000000000000000000000000000000..dd3d84bcb4739e49a00063d96cce2d2a6e6022bf Binary files /dev/null and b/doc/tutorials/merge_requests/img/script_failure_v18_3.png differ