From 098c69925e500b5b4a65f9416105bf2ae338d07b Mon Sep 17 00:00:00 2001 From: Tim Rizzi Date: Wed, 22 Oct 2025 09:55:45 -0700 Subject: [PATCH] Update single project workflow docs --- .../packages/workflows/project_registry.md | 513 +++++++++++++++--- 1 file changed, 449 insertions(+), 64 deletions(-) diff --git a/doc/user/packages/workflows/project_registry.md b/doc/user/packages/workflows/project_registry.md index 8adf3a8a5d0045..45ef3425ac2db3 100644 --- a/doc/user/packages/workflows/project_registry.md +++ b/doc/user/packages/workflows/project_registry.md @@ -2,7 +2,7 @@ stage: Package group: Package Registry info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments -title: Store all of your packages in one GitLab project +title: Centralize package management with dedicated registries --- {{< details >}} @@ -12,101 +12,486 @@ title: Store all of your packages in one GitLab project {{< /details >}} -Store packages from multiple sources in one project's package registry and configure your remote repositories to -point to this project in GitLab. +Organize your packages by type in dedicated projects within a centralized artifact management group. This approach provides the benefits of centralized package management while maintaining clear ownership and type-specific policies. Use this approach when you want to: -- Publish packages to GitLab in a different project than where your code is stored. -- Group packages together in one project (for example, all npm packages, all packages for a specific - department, or all private packages in the same project). -- Use one remote repository when installing packages for other projects. -- Migrate packages from a third-party package registry to a single location in GitLab. -- Have CI/CD pipelines build all packages to one project so you can manage packages in the same location. +- Organize packages by type with dedicated policies and settings +- Provide a single consumption endpoint for all organizational packages +- Migrate packages from third-party registries to a structured GitLab setup +- Separate package management concerns from application source code +- Apply different governance policies to different package types +- Maintain clear ownership while enabling organization-wide access -## Example walkthrough +## Recommended structure -Use each package management system -to publish different package types in the same place. +Create a dedicated top-level group for artifact management with projects organized by package type: -- - Watch a video of how to add Maven, npm, and Conan packages to [the same project](https://youtu.be/ui2nNBwN35c). -- [View an example project](https://gitlab.com/sabrams/my-package-registry/-/packages). +```plaintext +company/artifact-management/ (top-level group) +├── java-packages/ # Maven packages +├── node-packages/ # npm packages +├── python-packages/ # PyPI packages +├── docker-images/ # Container registry +├── terraform-modules/ # Terraform modules +├── nuget-packages/ # NuGet packages +└── generic-packages/ # Generic file packages +``` -## Store different package types in one GitLab project +> **Advanced separation**: Some organizations prefer additional separation based on package lifecycle or stability. For example, you might create separate projects for `java-releases/` and `java-snapshots/` to apply different cleanup policies, access controls, or approval workflows to stable versus development packages. -Let's take a look at how you might create one project to host all of your packages: +## Benefits of this approach -1. Create a new project in GitLab. The project doesn't require any code or content. -1. On the left sidebar, select **Project overview**, and note the project ID. -1. Create an access token for authentication. All package types in the package registry can be published by using: +**Centralized management:** - - A [personal access token](../../profile/personal_access_tokens.md). - - A [group access token](../../../user/group/settings/group_access_tokens.md) or [project access token](../../../user/project/settings/project_access_tokens.md). - - A [CI/CD job token](../../../ci/jobs/ci_job_token.md) (`CI_JOB_TOKEN`) in a CI/CD job. - The project's [job token allowlist](../../../ci/jobs/ci_job_token.md#add-a-group-or-project-to-the-job-token-allowlist) should list any projects publishing to this project's registry. +- Single group-level endpoint for consuming all package types +- Unified access control and governance policies +- Simplified token management and rotation - If the project is private, downloading packages requires authentication as well. +**Type-specific control:** -1. Configure your local project and publish the package. +- Dedicated lifecycle policies per package type +- Protected artifact rules tailored to each type +- Clear ownership and maintenance responsibilities -You can upload all package types to the same project, or -split up packages based on package type or visibility level. +**Scalable organization:** -### npm +- Easy to add new package types as projects +- Natural separation of concerns +- Aligns with most enterprise governance requirements -For npm packages: +## Set up the artifact management group -- Create an [`.npmrc` file](../npm_registry/_index.md#with-the-npmrc-file) to configure the registry URL. -- Scope your packages with the `publishConfig` option in the `package.json` file of your project. -- Publish packages with `npm publish`. +1. Create a new top-level group for artifact management: + 1. On the left sidebar, at the top, select **Create new** ({{< icon name="plus" >}}) and **New group**. + 1. In **Group name**, enter `Artifact Management` or similar. + 1. In **Group URL**, enter a path like `artifact-management`. + 1. Choose the appropriate [visibility level](../../public_access.md). + 1. Select **Create group**. -For more information, see [npm packages in the package registry](../npm_registry/_index.md). +2. Create projects for each package type you need: + 1. In your artifact management group, select **Create new** ({{< icon name="plus" >}}) and **New project/repository**. + 1. Select **Create blank project**. + 1. Enter a project name like `java-packages`, `node-packages`, etc. + 1. Set the appropriate visibility level. + 1. Select **Create project**. -### Maven +3. Configure group-level settings: + 1. In your artifact management group, on the left sidebar, select **Settings** > **General**. + 1. Configure any group-level policies like duplicate package prevention. + 1. Set up group access controls as needed. -For Maven packages: +## Configure authentication and access -1. Update your `pom.xml` file with `repository` and `distributionManagement` sections to configure the registry URL. -1. Add a `settings.xml` file and include your access token. -1. Publish packages with `mvn deploy`. +### Authentication patterns by use case -For more information, see [Maven packages in the package registry](../maven_repository/_index.md). +**For local development (developers):** -### Conan 1 +- Personal access tokens for individual developers +- Group access tokens for shared team credentials -For Conan 1: +**For CI/CD pipelines:** -- Add the GitLab package registry as a Conan registry remote. -- [Create your Conan 1 package](build_packages.md#build-a-conan-1-package) using the plus-separated (`+`) project path as your Conan user. For example, -if your project is located at `https://gitlab.com/foo/bar/my-proj`, -create your Conan package using `conan create . foo+bar+my-proj/channel`. `channel` is the package channel, such as `beta` or `stable`: +- CI/CD job tokens (preferred) - automatic authentication +- Project access tokens for special cases - ```shell - CONAN_LOGIN_USERNAME= CONAN_PASSWORD= conan upload MyPackage/1.0.0@foo+bar+my-proj/channel --all --remote=gitlab - ``` +**For external systems:** -- Publish your package with `conan upload` or your package recipe. +- Deploy tokens for read-only consumption +- Project/group access tokens for more granular control -For more information, see [Conan 1 packages in the package registry](../conan_1_repository/_index.md). +### Set up group-level access -### Conan 2 +1. **Create a group deploy token** for organization-wide package consumption: + 1. In your artifact management group, on the left sidebar, select **Settings** > **Repository**. + 1. Expand **Deploy tokens**. + 1. Complete the fields: + - **Name**: `package-consumption` + - **Scopes**: Select `read_package_registry` + 1. Select **Create deploy token**. + 1. Save the token securely - you'll use this for consuming packages. -For Conan 2: +2. **Configure job token allowlist** (if using CI/CD job tokens for publishing): + 1. In each package-specific project, on the left sidebar, select **Settings** > **CI/CD**. + 1. Expand **Token Access**. + 1. Add projects that should be allowed to publish packages to this registry. -- Add the GitLab package registry as a Conan registry remote. -- [Create your Conan 2 package](build_packages.md#conan-2). -- Publish your package with `conan upload` or your package recipe. +### Set up project-level settings -For more information, see [Conan 2 packages in the package registry](../conan_2_repository/_index.md). +For each package type project, configure: -### Composer +1. **Lifecycle policies** appropriate for that package type +1. **Protected packages** rules if needed +1. **Protected container images/tag** rules if needed +1. **Project access tokens** for specific use cases -You can't publish a Composer package outside of its project. Support for publishing Composer packages -in other projects is proposed in [issue 250633](https://gitlab.com/gitlab-org/gitlab/-/issues/250633). +## Publishing packages -### All other package types +Teams publish packages to the appropriate type-specific project registry. Here are examples for each supported package format: -[All package types supported by GitLab](../_index.md) can be published in -the same GitLab project. In previous releases, not all package types could -be published in the same project. +{{< tabs >}} + +{{< tab title="Maven" >}} + +Configure your project's `pom.xml` to publish to the java-packages project: + +```xml + + + gitlab-maven + ${CI_API_V4_URL}/projects/JAVA_PACKAGES_PROJECT_ID/packages/maven + + + gitlab-maven + ${CI_API_V4_URL}/projects/JAVA_PACKAGES_PROJECT_ID/packages/maven + + +``` + +Configure authentication in your `settings.xml`: + +```xml + + + gitlab-maven + + + + Job-Token + ${CI_JOB_TOKEN} + + + + + +``` + +Publish with: +```bash +mvn deploy +``` + +{{< /tab >}} + +{{< tab title="npm" >}} + +Configure your project's `package.json`: + +```json +{ + "name": "@company/my-package", + "publishConfig": { + "registry": "${CI_API_V4_URL}/projects/NODE_PACKAGES_PROJECT_ID/packages/npm/" + } +} +``` + +For CI/CD publishing, the job token is used automatically: + +```yaml +publish: + script: + - npm publish +``` + +For local publishing, configure authentication: + +```bash +npm config set @company:registry https://gitlab.example.com/api/v4/projects/NODE_PACKAGES_PROJECT_ID/packages/npm/ +npm config set //gitlab.example.com/api/v4/projects/NODE_PACKAGES_PROJECT_ID/packages/npm/:_authToken ${PERSONAL_ACCESS_TOKEN} +``` + +{{< /tab >}} + +{{< tab title="PyPI" >}} + +Configure publishing in your CI/CD pipeline: + +```yaml +publish: + script: + - pip install build twine + - python -m build + - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token twine upload --repository-url ${CI_API_V4_URL}/projects/PYTHON_PACKAGES_PROJECT_ID/packages/pypi dist/* +``` + +For local publishing: + +```bash +twine upload --repository-url https://gitlab.example.com/api/v4/projects/PYTHON_PACKAGES_PROJECT_ID/packages/pypi --username __token__ --password ${PERSONAL_ACCESS_TOKEN} dist/* +``` + +{{< /tab >}} + +{{< tab title="Container Registry" >}} + +Build and push Docker images: + +```yaml +build-image: + script: + - docker build -t $CI_REGISTRY/artifact-management/docker-images/my-app:$CI_COMMIT_SHA . + - docker push $CI_REGISTRY/artifact-management/docker-images/my-app:$CI_COMMIT_SHA +``` + +For local development: + +```bash +docker login gitlab.example.com -u ${USERNAME} -p ${PERSONAL_ACCESS_TOKEN} +docker push gitlab.example.com/artifact-management/docker-images/my-app:latest +``` + +{{< /tab >}} + +{{< tab title="Terraform" >}} + +Publish Terraform modules: + +```yaml +publish-module: + script: + - tar -czf module.tar.gz *.tf + - 'curl --header "JOB-TOKEN: $CI_JOB_TOKEN" --upload-file module.tar.gz "${CI_API_V4_URL}/projects/TERRAFORM_PACKAGES_PROJECT_ID/packages/terraform/modules/my-module/my-provider/1.0.0/file"' +``` + +{{< /tab >}} + +{{< tab title="NuGet" >}} + +Configure publishing in your project file or CI/CD: + +```yaml +publish: + script: + - dotnet pack + - dotnet nuget push "bin/Release/*.nupkg" --source ${CI_API_V4_URL}/projects/NUGET_PACKAGES_PROJECT_ID/packages/nuget/index.json --api-key ${CI_JOB_TOKEN} +``` + +For local publishing: + +```bash +dotnet nuget push package.nupkg --source https://gitlab.example.com/api/v4/projects/NUGET_PACKAGES_PROJECT_ID/packages/nuget/index.json --api-key ${PERSONAL_ACCESS_TOKEN} +``` + +{{< /tab >}} + +{{< tab title="Generic" >}} + +Upload generic packages: + +```yaml +upload-package: + script: + - 'curl --header "JOB-TOKEN: $CI_JOB_TOKEN" --upload-file my-package.zip "${CI_API_V4_URL}/projects/GENERIC_PACKAGES_PROJECT_ID/packages/generic/my-package/1.0.0/my-package.zip"' +``` + +{{< /tab >}} + +{{< /tabs >}} + +## Consuming packages + +Configure your projects to consume packages from the group-level endpoint. This provides access to all package types through a single configuration: + +{{< tabs >}} + +{{< tab title="Maven" >}} + +Configure your `pom.xml` to consume from the group registry: + +```xml + + + gitlab-maven + https://gitlab.example.com/api/v4/groups/artifact-management/-/packages/maven + + +``` + +Configure authentication in your `settings.xml`: + +```xml + + + + gitlab-maven + deploy-token-username + deploy-token-password + + + +``` + +{{< /tab >}} + +{{< tab title="npm" >}} + +Configure your `.npmrc` file: + +```ini +@company:registry=https://gitlab.example.com/api/v4/groups/artifact-management/-/packages/npm/ +//gitlab.example.com/api/v4/groups/artifact-management/-/packages/npm/:_authToken=${DEPLOY_TOKEN} +``` + +{{< /tab >}} + +{{< tab title="PyPI" >}} + +Configure pip to use the group registry: + +```ini +# pip.conf or ~/.pip/pip.conf +[global] +extra-index-url = https://deploy-token-username:deploy-token-password@gitlab.example.com/api/v4/groups/artifact-management/-/packages/pypi/simple/ +``` + +Or use environment variables: + +```bash +pip install --index-url https://deploy-token-username:deploy-token-password@gitlab.example.com/api/v4/groups/artifact-management/-/packages/pypi/simple/ --no-index my-package +``` + +{{< /tab >}} + +{{< tab title="Container Registry" >}} + +Pull images from the group registry: + +```bash +docker login gitlab.example.com -u deploy-token-username -p deploy-token-password +docker pull gitlab.example.com/artifact-management/docker-images/my-app:latest +``` + +{{< /tab >}} + +{{< tab title="Terraform" >}} + +Configure Terraform to use GitLab credentials via environment variables: + +```bash +export TF_TOKEN_gitlab_example_com="deploy-token-password" +``` + +Then reference modules in your Terraform configuration: + +```hcl +module "example" { + source = "gitlab.example.com/artifact-management/terraform-modules//my-module" + version = "1.0.0" +} +``` + +Or using the project-specific URL: + +```hcl +module "example" { + source = "https://gitlab.example.com/api/v4/projects/TERRAFORM_PACKAGES_PROJECT_ID/packages/terraform/modules/my-module/my-provider/1.0.0" +} +``` + +{{< /tab >}} + +{{< tab title="NuGet" >}} + +Configure NuGet to use the group registry: + +```xml + + + + + + + + + + + + + +``` + +{{< /tab >}} + +{{< tab title="Generic" >}} + +Download generic packages: + +```bash + +curl --header "DEPLOY-TOKEN: ${DEPLOY_TOKEN}" "https://gitlab.example.com/api/v4/groups/artifact-management/-/packages/generic/my-package/1.0.0/my-package.zip" --output my-package.zip +``` + +{{< /tab >}} + +{{< /tabs >}} + +## Example CI/CD configuration + +Here's an example of how a project might consume packages from multiple types: + +```yaml + +stages: + - build + - test + +variables: + MAVEN_OPTS: "-Dmaven.repo.local=${CI_PROJECT_DIR}/.m2/repository" + +before_script: + # Configure npm registry + - echo "@company:registry=${CI_API_V4_URL}/groups/artifact-management/-/packages/npm/" >> .npmrc + - echo "//${CI_SERVER_HOST}/api/v4/groups/artifact-management/-/packages/npm/:_authToken=${CI_JOB_TOKEN}" >> .npmrc + +build: + stage: build + script: + # Install npm dependencies from group registry + - npm install + # Build with Maven dependencies from group registry + - mvn compile + cache: + paths: + - .m2/repository/ + - node_modules/ +``` + +## Alternative: Publishing alongside source code + +Some organizations prefer publishing packages alongside their application source code, as described in the [enterprise scale tutorial](../tutorial_structure_package_registry_enterprise_scale/index.md). This approach works well when: + +- Packages are tightly coupled to specific applications +- You want package ownership to align with source code ownership +- Teams manage both code and packages together + +The artifact management approach described in this guide works better when: + +- You want centralized package governance +- Packages are shared across multiple projects +- You need type-specific policies and controls +- You're migrating from traditional artifact repositories + +## Summary + +The dedicated artifact management structure provides: + +**Simplified consumption:** + +- Single group endpoint for all package types +- Consistent authentication across package formats +- Easy discovery of organizational packages + +**Clear organization:** + +- Type-specific projects with appropriate policies +- Separation of package management from application code +- Scalable structure that grows with your needs + +**Flexible governance:** + +- Group-level policies for organization-wide rules +- Project-level controls for type-specific requirements +- Multiple authentication options for different use cases + +Start with the package types your organization uses most, then expand the structure as you adopt additional package formats. This approach scales naturally while maintaining security and ease of use. -- GitLab