diff --git a/doc/administration/gitlab_duo_self_hosted/_index.md b/doc/administration/gitlab_duo_self_hosted/_index.md index e0691b66d6e25b050ed4ead2e55d7e45b8faace6..f2dc10a65dbd0befad8f1757edb6a9743328dd4e 100644 --- a/doc/administration/gitlab_duo_self_hosted/_index.md +++ b/doc/administration/gitlab_duo_self_hosted/_index.md @@ -137,7 +137,7 @@ In a fully self-hosted configuration, you deploy your own AI gateway and use onl {{< alert type="note" >}} -This configuration only includes models configured through your self-hosted AI gateway. If you use [GitLab AI vendor models](configure_duo_features.md#configure-the-feature-to-use-a-gitlab-ai-vendor-model) for any features, those features will connect to the GitLab-hosted AI gateway instead of your self-hosted gateway, making it a hybrid configuration rather than fully self-hosted. +This configuration only includes models configured through your self-hosted AI gateway. If you use [GitLab AI vendor models](configure_duo_features.md#configure-a-feature-to-use-a-gitlab-ai-vendor-model) for any features, those features will connect to the GitLab-hosted AI gateway instead of your self-hosted gateway, making it a hybrid configuration rather than fully self-hosted. {{< /alert >}} @@ -194,7 +194,7 @@ For licensing, you must have a GitLab Premium or Ultimate subscription, and [Git For more information, see: -- [Configure GitLab AI vendor models](configure_duo_features.md#configure-the-feature-to-use-a-gitlab-ai-vendor-model) +- [Configure GitLab AI vendor models](configure_duo_features.md#configure-a-feature-to-use-a-gitlab-ai-vendor-model) #### GitLab managed models diff --git a/doc/administration/gitlab_duo_self_hosted/configure_duo_features.md b/doc/administration/gitlab_duo_self_hosted/configure_duo_features.md index 5dc3e7a18b1d8e4ec3b62bc4fc51d11c9129ab4e..bab0349097af37d0316cc3c1d8b61f96d6822105 100644 --- a/doc/administration/gitlab_duo_self_hosted/configure_duo_features.md +++ b/doc/administration/gitlab_duo_self_hosted/configure_duo_features.md @@ -38,12 +38,14 @@ To configure your GitLab instance to access the available self-hosted models in 1. Configure the self-hosted model. 1. Configure the GitLab Duo features to use your self-hosted model. -## Configure your GitLab instance to access the AI gateway +## Configure access to the local AI gateway + +To configure access between your GitLab instance and your local AI gateway: 1. On the left sidebar, at the bottom, select **Admin**. 1. Select **GitLab Duo**. 1. Select **Change configuration**. -1. Under **Local AI Gateway URL**, enter your AI Gateway URL. +1. Under **Local AI Gateway URL**, enter your AI gateway URL. 1. Select **Save changes**. {{< alert type="note" >}} @@ -73,35 +75,34 @@ For more information, see the history. {{< /alert >}} -You must provide a URL to access the Agent Platform service from your GitLab instance. - -- Prefix the URL for the Agent Platform service cannot start with `http://` or `https://`. - -- If the URL for the Agent Platform service is not set up with TLS, you must set the `DUO_AGENT_PLATFORM_SERVICE_SECURE` environment variable in your GitLab instance: - - - For Linux package installations, in `gitlab_rails['env']`, set `'DUO_AGENT_PLATFORM_SERVICE_SECURE' => false` - - For self-compiled installations, in `/etc/default/gitlab` set `export DUO_AGENT_PLATFORM_SERVICE_SECURE=false` +To access the Agent Platform service from your GitLab instance: 1. On the left sidebar, at the bottom, select **Admin**. 1. Select **GitLab Duo**. 1. Select **Change configuration**. 1. Under **Local URL for the GitLab Duo Agent Platform service**, enter the URL for the local Agent Platform service. + - The URL prefix cannot start with `http://` or `https://`. + + - If the URL is not set up with TLS, you must set the `DUO_AGENT_PLATFORM_SERVICE_SECURE` environment variable in your GitLab instance: + + - For Linux package installations, in `gitlab_rails['env']`, set `'DUO_AGENT_PLATFORM_SERVICE_SECURE' => false` + - For self-compiled installations, in `/etc/default/gitlab` set `export DUO_AGENT_PLATFORM_SERVICE_SECURE=false` 1. Select **Save changes**. -## Configure the self-hosted model +## Add a self-hosted model + +You must add a self-hosted model to your GitLab instance to use the model with GitLab Duo features. Prerequisites: - You must be an administrator. -- You must have a Premium or Ultimate license. -- You must have a GitLab Duo Enterprise license add-on. -To configure a self-hosted model: +To add a self-hosted model: 1. On the left sidebar, at the bottom, select **Admin**. 1. Select **GitLab Duo**. 1. Select **Configure GitLab Duo Self-Hosted**. - - If the **Configure GitLab Duo Self-Hosted** button is not available, synchronize your + - If the **Configure GitLab Duo Self-Hosted** is not available, synchronize your subscription after purchase: 1. On the left sidebar, select **Subscription**. 1. In **Subscription details**, to the right of **Last sync**, select @@ -112,7 +113,7 @@ To configure a self-hosted model: - **Model family**: Select the model family the deployment belongs to. You can select either a supported or compatible model. - **Endpoint**: Enter the URL where the model is hosted. - **API key**: Optional. Add an API key if you need one to access the model. - - **Model identifier**: This is a required field. The value of this field is based on your deployment method, and should match the following structure: + - **Model identifier**: Enter the model identifier based on your deployment method. The model identifier should match the following format: | Deployment method | Format | Example | |-------------|---------|---------| @@ -120,40 +121,40 @@ To configure a self-hosted model: | Amazon Bedrock | `bedrock/` | `bedrock/mistral.mixtral-8x7b-instruct-v0:1` | | Azure OpenAI | `azure/` | `azure/gpt-35-turbo` | - - For Amazon Bedrock models: +1. Select **Create self-hosted model**. - 1. Set your `AWS_REGION` and make sure you have access to models in that region in your AI gateway Docker configuration. - 1. Add the appropriate region prefix to the model's inference profile ID - for cross-region inferencing. - 1. Enter the region prefix and model inference profile ID in the **Model identifier** - field, with the `bedrock/` prefix. +For more information about configuring the endpoint and model identifier for models deployed through vLLM, see the [vLLM documentation](supported_llm_serving_platforms.md#find-the-model-name). - For example, for the Anthropic Claude 3.5 v2 model in the Tokyo region: +### Set the model identifier for Amazon Bedrock models - - The `AWS_REGION` is `ap-northeast-1`. - - The cross-region inferencing prefix is `apac.`. - - The model identifier is `bedrock/apac.anthropic.claude-3-5-sonnet-20241022-v2:0`. +To set a model identifer for an Amazon Bedrock model: - Some regions are not supported by cross-region inferencing. For these regions, the model identifier should be specified without the region prefix. For example: +1. Set your `AWS_REGION`. Ensure you have access to models in that region in your AI gateway Docker configuration. +1. Add the region prefix to the model's inference profile ID for cross-region inferencing. +1. Use the `bedrock/` prefix region as the prefix for the model identifier. - - The `AWS_REGION` is `eu-west-2`. - - The model identifier should be `bedrock/anthropic.claude-3-7-sonnet-20250219-v1:0`. + For example, for the Anthropic Claude 3.5 v2 model in the Tokyo region: -1. Select **Create self-hosted model**. + - The `AWS_REGION` is `ap-northeast-1`. + - The cross-region inferencing prefix is `apac.`. + - The model identifier is `bedrock/apac.anthropic.claude-3-5-sonnet-20241022-v2:0`. -For more information about: +Some regions are not supported by cross-region inferencing. For these regions, do not specify a region prefix in the model identifier. For example: -- Configuring the endpoint and model identifier for models deployed through vLLM, see the [vLLM documentation](supported_llm_serving_platforms.md#find-the-model-name). -- Configuring Amazon Bedrock models with cross-region inferencing, see the - [Amazon supported regions and models for inference profiles documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html) +- The `AWS_REGION` is `eu-west-2`. +- The model identifier is `bedrock/anthropic.claude-3-7-sonnet-20250219-v1:0`. ## Configure self-hosted beta models and features Prerequisites: - You must be an administrator. -- You must have an Premium or Ultimate license. -- You must have a GitLab Duo Enterprise license add-on. + +{{< alert type="note" >}} + +Turning on beta self-hosted models and features also accepts the [GitLab Testing Agreement](https://handbook.gitlab.com/handbook/legal/testing-agreement/). + +{{< /alert >}} To enable self-hosted beta models and features: @@ -163,19 +164,11 @@ To enable self-hosted beta models and features: 1. Under **Self-hosted beta models and features**, select the **Use beta models and features in GitLab Duo Self-Hosted** checkbox. 1. Select **Save changes**. -{{< alert type="note" >}} - -Turning on beta self-hosted models and features also accepts the [GitLab Testing Agreement](https://handbook.gitlab.com/handbook/legal/testing-agreement/). - -{{< /alert >}} - ## Configure GitLab Duo features to use self-hosted models Prerequisites: - You must be an administrator. -- You must have an Premium or Ultimate license. -- You must have a GitLab Duo Enterprise license add-on. ### View configured features @@ -191,9 +184,9 @@ subscription after purchase: 1. In **Subscription details**, to the right of **Last sync**, select synchronize subscription ({{< icon name="retry" >}}). -### Configure the feature to use a self-hosted model +### Configure a feature to use a self-hosted model -Configure the GitLab Duo feature and sub-feature to send queries to the configured self-hosted model: +Configure a GitLab Duo feature and sub-feature to send queries to the self-hosted model: 1. On the left sidebar, at the bottom, select **Admin**. 1. Select **GitLab Duo**. @@ -201,11 +194,18 @@ Configure the GitLab Duo feature and sub-feature to send queries to the configur 1. Select the **AI-native features** tab. 1. For the feature and sub-feature you want to configure, from the dropdown list, choose the self-hosted model you want to use. - For example, for the code generation sub-feature under GitLab Duo Code Suggestions, you can select **Claude-3 on Bedrock deployment (Claude 3)**. + For example, for the code generation, you can select **Claude-3 on Bedrock deployment (Claude 3)**. ![GitLab Duo Self-Hosted Feature Configuration](img/gitlab_duo_self_hosted_feature_configuration_v17_11.png) -### Configure the feature to use a GitLab AI vendor model +{{< alert type="note" >}} + +If you don't specify a model for a GitLab Duo Chat sub-feature, it automatically uses the model configured for **General Chat**. +This ensures all Chat functionality works without requiring individual model configuration for each sub-feature. + +{{< /alert >}} + +### Configure a feature to use a GitLab AI vendor model {{< details >}} @@ -226,7 +226,7 @@ For more information, see the history. {{< /alert >}} -In GitLab 18.3 and later, even when you are using your self-hosted AI gateway and models, you can configure a specific GitLab Duo feature to use a GitLab AI vendor model. +You can configure a GitLab Duo feature to use the GitLab AI vendor model, even if you use a self-hosted AI gateway and models. 1. On the left sidebar, at the bottom, select **Admin**. 1. Select **GitLab Duo**. @@ -234,23 +234,11 @@ In GitLab 18.3 and later, even when you are using your self-hosted AI gateway an 1. Select the **AI-native features** tab. 1. For the feature and sub-feature you want to configure, from the dropdown list, select **GitLab AI vendor model**. - For example, for the code generation sub-feature under GitLab Duo Code Suggestions, you can select **GitLab AI vendor model**. - - ![GitLab Duo Self-Hosted Feature Configuration using GitLab AI vendor model](img/gitlab_duo_self_hosted_feature_configuration_with_vendored_model_v18_3.png) - -For more information on this hybrid configuration, see the documentation on [GitLab Duo Self-Hosted configuration type](_index.md#configuration-types). - -### GitLab Duo Chat sub-feature fall back configuration - -When configuring GitLab Duo Chat sub-features, if you do not select a specific model for a sub-feature, that sub-feature automatically falls back to using the model configured for **General Chat**. This ensures all Chat functionality works even if you have not explicitly configured each sub-feature with its own model. - -### Self-host the GitLab documentation - -If your setup of GitLab Duo Self-Hosted stops you from accessing the GitLab documentation at `docs.gitlab.com`, you can self-host the documentation instead. For more information, see how to [host the GitLab product documentation](../docs_self_host.md). +![GitLab Duo Self-Hosted feature configuration using GitLab AI vendor model](img/gitlab_duo_self_hosted_feature_configuration_with_vendored_model_v18_3.png) ### Disable GitLab Duo features -GitLab Duo features remain turned on even if you have not chosen a model for a sub-feature. If you don't select a model for a +GitLab Duo features remain turned on even if you have not chosen a model for a sub-feature. If you don't select a model for a sub-feature of GitLab Duo Chat, it uses the model configured for **General Chat** instead. To disable a GitLab Duo feature or sub-feature: @@ -261,12 +249,14 @@ To disable a GitLab Duo feature or sub-feature: 1. Select the **AI-native features** tab. 1. For the feature or sub-feature you want to disable, from the dropdown list, select **Disabled**. - For example, to specifically disable the `Write Test` and `Refactor Code` features, select **Disabled**: +![Disabling GitLab Duo Feature](img/gitlab_duo_self_hosted_disable_feature_v17_11.png) + +### Self-host the GitLab documentation - ![Disabling GitLab Duo Feature](img/gitlab_duo_self_hosted_disable_feature_v17_11.png) +If your setup of GitLab Duo Self-Hosted stops you from accessing the GitLab documentation at `docs.gitlab.com`, you can self-host the documentation instead. For more information, see how to [host the GitLab product documentation](../docs_self_host.md). ## Related topics - [Supported models](supported_models_and_hardware_requirements.md#supported-models) - [Compatible models](supported_models_and_hardware_requirements.md#compatible-models) -- [GitLab Duo Self-Hosted configuration types](_index.md#configuration-types) +- [GitLab Duo Self-Hosted configuration types](_index.md#configuration-types) \ No newline at end of file diff --git a/doc/administration/gitlab_duo_self_hosted/supported_models_and_hardware_requirements.md b/doc/administration/gitlab_duo_self_hosted/supported_models_and_hardware_requirements.md index 7a185fdf481d089b9f4cfe45ab8cf989bb0327b0..6c0ad580397e1116a67bdc1a9a1ce666612fba87 100644 --- a/doc/administration/gitlab_duo_self_hosted/supported_models_and_hardware_requirements.md +++ b/doc/administration/gitlab_duo_self_hosted/supported_models_and_hardware_requirements.md @@ -130,7 +130,7 @@ For more information, see the history. GitLab AI vendor models integrate with GitLab-hosted AI gateway infrastructure to provide access to AI models curated and made available by GitLab. Instead of using your own self-hosted models, you can choose to use GitLab AI vendor models for specific GitLab Duo features. -To choose which features use GitLab AI vendor models, see [Configure GitLab AI vendor models](configure_duo_features.md#configure-the-feature-to-use-a-gitlab-ai-vendor-model). +To choose which features use GitLab AI vendor models, see [Configure GitLab AI vendor models](configure_duo_features.md#configure-a-feature-to-use-a-gitlab-ai-vendor-model). When enabled for a specific feature: diff --git a/doc/administration/gitlab_duo_self_hosted/troubleshooting.md b/doc/administration/gitlab_duo_self_hosted/troubleshooting.md index 0df2040bef05e965a0661b0f9c192e8c15f6fc4f..635d288ee3c6eccc7d2264418efacd061caed2f3 100644 --- a/doc/administration/gitlab_duo_self_hosted/troubleshooting.md +++ b/doc/administration/gitlab_duo_self_hosted/troubleshooting.md @@ -191,7 +191,7 @@ To check that the AI gateway URL is correct, run the following on the GitLab Rai Ai::Setting.instance.ai_gateway_url == "" ``` -If the AI gateway is not set up, [configure your GitLab instance to access the AI gateway](configure_duo_features.md#configure-your-gitlab-instance-to-access-the-ai-gateway). +If the AI gateway is not set up, [configure your GitLab instance to access the AI gateway](configure_duo_features.md#configure-to-access-the-local-ai-gateway). ## Validate the GitLab Duo Agent Platform service URL diff --git a/doc/install/install_ai_gateway.md b/doc/install/install_ai_gateway.md index 28fa09a19ac58fae0e1ad279786b22950fe557d7..41b88592c28474e63410b0c50bc1542642215ba1 100644 --- a/doc/install/install_ai_gateway.md +++ b/doc/install/install_ai_gateway.md @@ -82,7 +82,7 @@ Using the nightly version is **not recommended** because it can cause incompatib From the container host, accessing `http://localhost:5052` should return `{"error":"No authorization header presented"}`. 1. Ensure that ports `5052` and `50052` are forwarded to the container from the host. -1. Configure the [AI gateway URL](../administration/gitlab_duo_self_hosted/configure_duo_features.md#configure-your-gitlab-instance-to-access-the-ai-gateway) and the [GitLab Duo Agent Platform service URL](../administration/gitlab_duo_self_hosted/configure_duo_features.md#configure-access-to-the-gitlab-duo-agent-platform). +1. Configure the [AI gateway URL](../administration/gitlab_duo_self_hosted/configure_duo_features.md#configure-access-the-ai-gateway) and the [GitLab Duo Agent Platform service URL](../administration/gitlab_duo_self_hosted/configure_duo_features.md#configure-access-to-the-gitlab-duo-agent-platform). 1. If you are going to use your own self-hosted model for GitLab Duo Agent Platform, and the URL is not set up with TLS, you must set the `DUO_AGENT_PLATFORM_SERVICE_SECURE` environment variable in your GitLab instance: - For Linux package installations, in `gitlab_rails['env']`, set `'DUO_AGENT_PLATFORM_SERVICE_SECURE' => false` @@ -269,7 +269,7 @@ To deploy and validate the solution: docker ps ``` -1. Configure your [GitLab instance to access the AI gateway](../administration/gitlab_duo_self_hosted/configure_duo_features.md#configure-your-gitlab-instance-to-access-the-ai-gateway). +1. Configure your [GitLab instance to access the AI gateway](../administration/gitlab_duo_self_hosted/configure_duo_features.md#configure-to-access-the-local-ai-gateway). 1. Configure your GitLab instance to access the URL for the [GitLab Duo Agent Platform service](../administration/gitlab_duo_self_hosted/configure_duo_features.md#configure-access-to-the-gitlab-duo-agent-platform).