From 62cb0e4390a58233b16bb5ea98814718f7f393d2 Mon Sep 17 00:00:00 2001 From: Wei-Meng Lee Date: Mon, 30 Mar 2020 16:32:38 +0000 Subject: [PATCH 1/8] Split scaling and HA in PostgreSQL docs --- doc/administration/database/index.md | 100 ++++++++++++++++++ .../high_availability/database.md | 83 +-------------- 2 files changed, 101 insertions(+), 82 deletions(-) create mode 100644 doc/administration/database/index.md diff --git a/doc/administration/database/index.md b/doc/administration/database/index.md new file mode 100644 index 00000000000000..6445846d8d9e8f --- /dev/null +++ b/doc/administration/database/index.md @@ -0,0 +1,100 @@ +--- +type: reference +--- + +# Database + +GitLab supports only the use of PostgreSQL as its database. + +NOTE: **Note:** +Prior to GitLab 12.1, MySQL was supported for Enterprise Edition licensed +instances. If you're using MySQL, you need to +[migrate to PostgreSQL](../../update/mysql_to_postgresql.html) +before upgrade to 12.1 and beyond. + +## Requirements + +Omnibus GitLab comes packaged with PostgreSQL configured with the appropriate +extensions and ready to go out of the box. + +If you are [installing GitLab from source](https://docs.gitlab.com/ee/install/installation.html) +or choose to provide your own database, PostgreSQL requirements can be found in +GitLab [system requirements documentation](https://docs.gitlab.com/ee/install/requirements.html#postgresql-requirements). + +## Provide your own PostgreSQL instance **(CORE ONLY)** + +If you're hosting GitLab on a cloud provider, you can optionally use a +managed service for PostgreSQL. For example, AWS offers a managed Relational +Database Service (RDS) that runs PostgreSQL. + +If you use a cloud-managed service, or provide your own PostgreSQL: + +1. Set up PostgreSQL according to the + [database requirements document](../../install/requirements.md#database). +1. Set up a `gitlab` username with a password of your choice. The `gitlab` user + needs privileges to create the `gitlabhq_production` database. +1. Configure the GitLab application servers with the appropriate details. + This step is covered in [Configuring GitLab for HA](gitlab.md). + +## Standalone PostgreSQL server using Omnibus GitLab **(CORE ONLY)** + +You can use the GitLab Omnibus package to easily +deploy the bundled PostgreSQL. + +1. SSH into the PostgreSQL server. +1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab + package you want using **steps 1 and 2** from the GitLab downloads page. + - Do not complete any other steps on the download page. +1. Generate a password hash for PostgreSQL. This assumes you will use the default + username of `gitlab` (recommended). The command will request a password + and confirmation. Use the value that is output by this command in the next + step as the value of `POSTGRESQL_PASSWORD_HASH`. + + ```shell + sudo gitlab-ctl pg-password-md5 gitlab + ``` + +1. Edit `/etc/gitlab/gitlab.rb` and add the contents below, updating placeholder + values appropriately. + + - `POSTGRESQL_PASSWORD_HASH` - The value output from the previous step + - `APPLICATION_SERVER_IP_BLOCKS` - A space delimited list of IP subnets or IP + addresses of the GitLab application servers that will connect to the + database. Example: `%w(123.123.123.123/32 123.123.123.234/32)` + + ```ruby + # Disable all components except PostgreSQL + roles ['postgres_role'] + repmgr['enable'] = false + consul['enable'] = false + prometheus['enable'] = false + alertmanager['enable'] = false + pgbouncer_exporter['enable'] = false + redis_exporter['enable'] = false + gitlab_exporter['enable'] = false + + postgresql['listen_address'] = '0.0.0.0' + postgresql['port'] = 5432 + + # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value + postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH' + + # Replace XXX.XXX.XXX.XXX/YY with Network Address + # ???? + postgresql['trust_auth_cidr_addresses'] = %w(APPLICATION_SERVER_IP_BLOCKS) + + # Disable automatic database migrations + gitlab_rails['auto_migrate'] = false + ``` + + NOTE: **Note:** The role `postgres_role` was introduced with GitLab 10.3 + +1. [Reconfigure GitLab] for the changes to take effect. +1. Note the PostgreSQL node's IP address or hostname, port, and + plain text password. These will be necessary when configuring the GitLab + application servers later. +1. [Enable monitoring](#enable-monitoring) + +Advanced configuration options are supported and can be added if +needed. + diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index 4362ed3e64b24c..ecb02afd5755e4 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -2,96 +2,15 @@ type: reference --- -# Configuring PostgreSQL for Scaling and High Availability +# Configuring PostgreSQL for High Availability In this section, you'll be guided through configuring a PostgreSQL database to be used with GitLab in a highly available environment. -## Provide your own PostgreSQL instance **(CORE ONLY)** - -If you're hosting GitLab on a cloud provider, you can optionally use a -managed service for PostgreSQL. For example, AWS offers a managed Relational -Database Service (RDS) that runs PostgreSQL. - -If you use a cloud-managed service, or provide your own PostgreSQL: - -1. Set up PostgreSQL according to the - [database requirements document](../../install/requirements.md#database). -1. Set up a `gitlab` username with a password of your choice. The `gitlab` user - needs privileges to create the `gitlabhq_production` database. -1. Configure the GitLab application servers with the appropriate details. - This step is covered in [Configuring GitLab for HA](gitlab.md). - ## PostgreSQL in a Scaled and Highly Available Environment This section is relevant for [Scalable and Highly Available Setups](README.md). -### Provide your own PostgreSQL instance **(CORE ONLY)** - -If you want to use your own deployed PostgreSQL instance(s), -see [Provide your own PostgreSQL instance](#provide-your-own-postgresql-instance-core-only) -for more details. However, you can use the GitLab Omnibus package to easily -deploy the bundled PostgreSQL. - -### Standalone PostgreSQL using GitLab Omnibus **(CORE ONLY)** - -1. SSH into the PostgreSQL server. -1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab - package you want using **steps 1 and 2** from the GitLab downloads page. - - Do not complete any other steps on the download page. -1. Generate a password hash for PostgreSQL. This assumes you will use the default - username of `gitlab` (recommended). The command will request a password - and confirmation. Use the value that is output by this command in the next - step as the value of `POSTGRESQL_PASSWORD_HASH`. - - ```shell - sudo gitlab-ctl pg-password-md5 gitlab - ``` - -1. Edit `/etc/gitlab/gitlab.rb` and add the contents below, updating placeholder - values appropriately. - - - `POSTGRESQL_PASSWORD_HASH` - The value output from the previous step - - `APPLICATION_SERVER_IP_BLOCKS` - A space delimited list of IP subnets or IP - addresses of the GitLab application servers that will connect to the - database. Example: `%w(123.123.123.123/32 123.123.123.234/32)` - - ```ruby - # Disable all components except PostgreSQL - roles ['postgres_role'] - repmgr['enable'] = false - consul['enable'] = false - prometheus['enable'] = false - alertmanager['enable'] = false - pgbouncer_exporter['enable'] = false - redis_exporter['enable'] = false - gitlab_exporter['enable'] = false - - postgresql['listen_address'] = '0.0.0.0' - postgresql['port'] = 5432 - - # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value - postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH' - - # Replace XXX.XXX.XXX.XXX/YY with Network Address - # ???? - postgresql['trust_auth_cidr_addresses'] = %w(APPLICATION_SERVER_IP_BLOCKS) - - # Disable automatic database migrations - gitlab_rails['auto_migrate'] = false - ``` - - NOTE: **Note:** The role `postgres_role` was introduced with GitLab 10.3 - -1. [Reconfigure GitLab] for the changes to take effect. -1. Note the PostgreSQL node's IP address or hostname, port, and - plain text password. These will be necessary when configuring the GitLab - application servers later. -1. [Enable monitoring](#enable-monitoring) - -Advanced configuration options are supported and can be added if -needed. - ### High Availability with GitLab Omnibus **(PREMIUM ONLY)** > Important notes: -- GitLab From 7821b81c3e74fcc9356e5a06e8b9b1ae39d82455 Mon Sep 17 00:00:00 2001 From: Wei-Meng Lee Date: Tue, 31 Mar 2020 09:45:23 +0000 Subject: [PATCH 2/8] Further doc consolidation --- doc/administration/database/index.md | 7 +++++-- doc/administration/external_database.md | 16 +--------------- doc/administration/high_availability/database.md | 8 ++++++++ 3 files changed, 14 insertions(+), 17 deletions(-) diff --git a/doc/administration/database/index.md b/doc/administration/database/index.md index 6445846d8d9e8f..5e7169f44dec1a 100644 --- a/doc/administration/database/index.md +++ b/doc/administration/database/index.md @@ -17,9 +17,9 @@ before upgrade to 12.1 and beyond. Omnibus GitLab comes packaged with PostgreSQL configured with the appropriate extensions and ready to go out of the box. -If you are [installing GitLab from source](https://docs.gitlab.com/ee/install/installation.html) +If you are [installing GitLab from source](../../install/installation.html) or choose to provide your own database, PostgreSQL requirements can be found in -GitLab [system requirements documentation](https://docs.gitlab.com/ee/install/requirements.html#postgresql-requirements). +GitLab [system requirements documentation](../../install/requirements.html#postgresql-requirements). ## Provide your own PostgreSQL instance **(CORE ONLY)** @@ -27,6 +27,9 @@ If you're hosting GitLab on a cloud provider, you can optionally use a managed service for PostgreSQL. For example, AWS offers a managed Relational Database Service (RDS) that runs PostgreSQL. +Alternatively, you may opt to manage your own PostgreSQL instance or cluster +separate from the GitLab Omnibus package. + If you use a cloud-managed service, or provide your own PostgreSQL: 1. Set up PostgreSQL according to the diff --git a/doc/administration/external_database.md b/doc/administration/external_database.md index ec2d30c82d1f6d..e50761d98cacaa 100644 --- a/doc/administration/external_database.md +++ b/doc/administration/external_database.md @@ -1,17 +1,3 @@ # Configure GitLab using an external PostgreSQL service -If you're hosting GitLab on a cloud provider, you can optionally use a -managed service for PostgreSQL. For example, AWS offers a managed Relational -Database Service (RDS) that runs PostgreSQL. - -Alternatively, you may opt to manage your own PostgreSQL instance or cluster -separate from the GitLab Omnibus package. - -If you use a cloud-managed service, or provide your own PostgreSQL instance: - -1. Set up PostgreSQL according to the - [database requirements document](../install/requirements.md#database). -1. Set up a `gitlab` username with a password of your choice. The `gitlab` user - needs privileges to create the `gitlabhq_production` database. -1. Configure the GitLab application servers with the appropriate details. - This step is covered in [Configuring GitLab for HA](high_availability/gitlab.md). +This content has been moved to the [database administration page](database/index.md). diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index ecb02afd5755e4..04c6dbc47d0e79 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -7,10 +7,18 @@ type: reference In this section, you'll be guided through configuring a PostgreSQL database to be used with GitLab in a highly available environment. +## Provide your own PostgreSQL instance **(CORE ONLY)** + +This content has been moved to the [database administration page](../database/index.md). + ## PostgreSQL in a Scaled and Highly Available Environment This section is relevant for [Scalable and Highly Available Setups](README.md). +### Standalone PostgreSQL using GitLab Omnibus **(CORE ONLY)** + +This content has been moved to the [database administration page](../database/index.md). + ### High Availability with GitLab Omnibus **(PREMIUM ONLY)** > Important notes: -- GitLab From 05d005f61855e2baca946eb1523cfbfa65e11969 Mon Sep 17 00:00:00 2001 From: Wei-Meng Lee Date: Tue, 31 Mar 2020 10:22:53 +0000 Subject: [PATCH 3/8] Consolidate external database docs --- .../database/external_database.md | 75 +++++++++++++++++++ doc/administration/database/index.md | 25 +------ 2 files changed, 76 insertions(+), 24 deletions(-) create mode 100644 doc/administration/database/external_database.md diff --git a/doc/administration/database/external_database.md b/doc/administration/database/external_database.md new file mode 100644 index 00000000000000..27b639c38f0116 --- /dev/null +++ b/doc/administration/database/external_database.md @@ -0,0 +1,75 @@ +--- +type: reference +--- + +# Using an external PostgreSQL service with GitLab + +If you're hosting GitLab on a cloud provider, you can optionally use a +managed service for PostgreSQL. For example, AWS offers a managed Relational +Database Service (RDS) that runs PostgreSQL. + +Alternatively, you may opt to manage your own PostgreSQL instance or cluster +separate from the GitLab Omnibus package. + +## Requirements + +Your external PostgreSQL service needs to be set up according to the GitLab +[database requirements documentation](../../install/requirements.html#postgresql-requirements). + +You will also need to set up a user (Omnibus GitLab uses `gitlab` by default) +with a password of your choice. This user needs privileges to create the +`gitlabhq_production` database. + +## Configure application services + +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['db_adapter'] = 'postgresql' + gitlab_rails['db_encoding'] = 'utf8' + gitlab_rails['db_host'] = '10.1.0.5' # IP/hostname of database server + gitlab_rails['db_port'] = 5432 + gitlab_rails['db_username'] = 'gitlab' + gitlab_rails['db_password'] = 'database_password' + ``` + +1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) + GitLab for the changes to take effect. + +**For installations from source** + +Follow the [Configure GitLab DB Settings](../install/installation.html#configure-gitlab-db-settings) section in the +installation from source documentation. + +## Initialize the database (for fresh installs only) + +CAUTION: **Caution:** +This will drop your existing database and recreate it as though a fresh GitLab +install. Do not run it on a database with existing production data! + +**For Omnibus installations** + +Omnibus GitLab will not automatically seed your external database. Run the +following command to import the database schema and create the first admin user: + +```shell +sudo gitlab-rake gitlab:setup +``` + +**For installations from source** + +Follow the [Initialize Database](../install/installation.html#initialize-database-and-activate-advanced-features) section in the +installation from source documentation. + +## Upgrading the external PostgreSQL service + +From time to time, GitLab will require the use of a more recent major version +of PostgreSQL. When upgrading your external PostgreSQL service, you will also +need to run the `ANALYZE VERBOSE;` query against your database to recreate +query plans. + +CAUTION: **Caution:** +If you neglect to do so, you may see extremely high (near 100%) CPU utilization +following a major PostgreSQL version upgrade. diff --git a/doc/administration/database/index.md b/doc/administration/database/index.md index 5e7169f44dec1a..1d9c1ac040dcf5 100644 --- a/doc/administration/database/index.md +++ b/doc/administration/database/index.md @@ -12,32 +12,9 @@ instances. If you're using MySQL, you need to [migrate to PostgreSQL](../../update/mysql_to_postgresql.html) before upgrade to 12.1 and beyond. -## Requirements - -Omnibus GitLab comes packaged with PostgreSQL configured with the appropriate -extensions and ready to go out of the box. - -If you are [installing GitLab from source](../../install/installation.html) -or choose to provide your own database, PostgreSQL requirements can be found in -GitLab [system requirements documentation](../../install/requirements.html#postgresql-requirements). - ## Provide your own PostgreSQL instance **(CORE ONLY)** -If you're hosting GitLab on a cloud provider, you can optionally use a -managed service for PostgreSQL. For example, AWS offers a managed Relational -Database Service (RDS) that runs PostgreSQL. - -Alternatively, you may opt to manage your own PostgreSQL instance or cluster -separate from the GitLab Omnibus package. - -If you use a cloud-managed service, or provide your own PostgreSQL: - -1. Set up PostgreSQL according to the - [database requirements document](../../install/requirements.md#database). -1. Set up a `gitlab` username with a password of your choice. The `gitlab` user - needs privileges to create the `gitlabhq_production` database. -1. Configure the GitLab application servers with the appropriate details. - This step is covered in [Configuring GitLab for HA](gitlab.md). +It is possible to use GitLab with an [external PostgreSQL service](external_database.md). ## Standalone PostgreSQL server using Omnibus GitLab **(CORE ONLY)** -- GitLab From eda79baf92d2ef0b8640762514edf10bdd13f548 Mon Sep 17 00:00:00 2001 From: Wei-Meng Lee Date: Tue, 31 Mar 2020 13:33:07 +0000 Subject: [PATCH 4/8] Add more changes --- .../database/app_configuration.md | 102 ++++++++++++++++++ .../database/external_database.md | 55 +++++++++- doc/administration/database/index.md | 91 ++++++++++++++++ 3 files changed, 245 insertions(+), 3 deletions(-) create mode 100644 doc/administration/database/app_configuration.md diff --git a/doc/administration/database/app_configuration.md b/doc/administration/database/app_configuration.md new file mode 100644 index 00000000000000..239f6c856ec133 --- /dev/null +++ b/doc/administration/database/app_configuration.md @@ -0,0 +1,102 @@ +--- +type: reference +--- + +# Application services client database connection settings + +The Unicorn and Sidekiq application services are the primary database clients +in a GitLab instance. + +## Configuring statement timeout + +The amount of time that Unicorn or Sidekiq will wait for a database transaction +to complete before timing out can now be adjusted: + +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['db_statement_timeout'] = 45000 # Specified in milliseconds + ``` + +1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) + GitLab for the changes to take effect. + +If `gitlab_rails['db_statement_timeout']` is not specified, Omnibus GitLab +uses the value of `postgresql['statement_timeout']` if present. Otherwise, a +default value of `60000` (60 seconds) is used. + +## Connecting to a PostgreSQL service over TCP/IP + +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['db_adapter'] = 'postgresql' + gitlab_rails['db_encoding'] = 'utf8' + gitlab_rails['db_host'] = '10.1.0.5' # IP or hostname of PostgreSQL server + gitlab_rails['db_port'] = 5432 + gitlab_rails['db_username'] = 'gitlab' + gitlab_rails['db_password'] = 'database_password' + ``` + +1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) + GitLab for the changes to take effect. + +**For installations from source** + +Follow the [Configure GitLab DB Settings](../install/installation.html#configure-gitlab-db-settings) section in the +installation from source documentation. + +### Verify server SSL certificate against CA bundle + +The application services can be configured to verify the PostgreSQL server +certificate against a CA bundle to prevent spoofing. + +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['db_sslmode'] = 'verify-full' + gitlab_rails['db_sslrootcert'] = 'your-full-ca-bundle.pem' + ``` + + NOTE: **Note:** + The CA bundle that is specified in `gitlab_rails['db_sslrootcert']` must + contain both the root and intermediate certificates. + + NOTE: **Note:** + If you are using Amazon RDS for your PostgreSQL server, ensure you download + and use the [combined CA bundle](https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem) + for `gitlab_rails['db_sslrootcert']`. More information on this can be found + in the [Using SSL to Encrypt a Connection to a DB instance](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html) + article on AWS. + +1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) + GitLab to apply the configuration changes. + +1. Restart PostgreSQL for the changes to take effect: + + ```shell + sudo gitlab-ctl restart postgresql + ``` + +## Disable automatic database migration + +If you have multiple GitLab servers sharing a database, you will want to limit +the number of nodes that are performing the migration steps during +reconfiguration. To do this: + +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['auto_migrate'] = false + ``` + +The next time a reconfigure is triggered, the migration steps will not be +performed. diff --git a/doc/administration/database/external_database.md b/doc/administration/database/external_database.md index 27b639c38f0116..04c46f37d8a0fe 100644 --- a/doc/administration/database/external_database.md +++ b/doc/administration/database/external_database.md @@ -11,7 +11,9 @@ Database Service (RDS) that runs PostgreSQL. Alternatively, you may opt to manage your own PostgreSQL instance or cluster separate from the GitLab Omnibus package. -## Requirements +## Setup + +### Requirements Your external PostgreSQL service needs to be set up according to the GitLab [database requirements documentation](../../install/requirements.html#postgresql-requirements). @@ -20,7 +22,7 @@ You will also need to set up a user (Omnibus GitLab uses `gitlab` by default) with a password of your choice. This user needs privileges to create the `gitlabhq_production` database. -## Configure application services +### Configure application client services **For Omnibus installations** @@ -43,7 +45,12 @@ with a password of your choice. This user needs privileges to create the Follow the [Configure GitLab DB Settings](../install/installation.html#configure-gitlab-db-settings) section in the installation from source documentation. -## Initialize the database (for fresh installs only) +NOTE: **Note:** +Additional application client service database connection options can be found +in the [application services database client configuration](app_configuration.md) +documentation. + +### Initialize the database (for fresh installs only) CAUTION: **Caution:** This will drop your existing database and recreate it as though a fresh GitLab @@ -63,6 +70,48 @@ sudo gitlab-rake gitlab:setup Follow the [Initialize Database](../install/installation.html#initialize-database-and-activate-advanced-features) section in the installation from source documentation. +## Backup and restore using rake task + +When using the [rake backup create and restore task](../../raketasks/backup_restore.md#create-a-backup-of-the-gitlab-system), +GitLab will attempt to use the packaged `pg_dump` command to craete a database +backup file and the packaged `psql` command to restore a backup. This will only +work if they are the correct versions. To check the versions of the packaged +`pg_dump` and `psql`: + +```shell +/opt/gitlab/embedded/bin/pg_dump --version +/opt/gitlab/embedded/bin/psql --version +``` + +If these versions are different from your external PostgreSQL service, you will +need to install tools that match your database version and then follow the +steps below. There are multiple ways to install PostgreSQL client tools. See +https://www.postgresql.org/download/ for options. + +Once the correct `psql` and `pg_dump` tools are available on your system, +follow these steps, using the correct path to the location you installed the +new tools: + +1. Add symbolic links to the non-packaged versions: + + ```shell + ln -s /path/to/new/pg_dump /path/to/new/psql /opt/gitlab/bin/ + ``` + +1. Check the versions: + + ```shell + /opt/gitlab/bin/pg_dump --version + /opt/gitlab/bin/psql --version + ``` + + They should now be the same as your external PostgreSQL service. + +After this is done, ensure that the backup and restore tasks are using the +correct executables by running both the [backup](../../raketasks/backup_restore.md#create-a-backup-of-the-gitlab-system) +and [restore](../../raketasks/backup_restore.html#restore-a-previously-created-backup) +tasks. + ## Upgrading the external PostgreSQL service From time to time, GitLab will require the use of a more recent major version diff --git a/doc/administration/database/index.md b/doc/administration/database/index.md index 1d9c1ac040dcf5..b004d4c4330603 100644 --- a/doc/administration/database/index.md +++ b/doc/administration/database/index.md @@ -12,6 +12,97 @@ instances. If you're using MySQL, you need to [migrate to PostgreSQL](../../update/mysql_to_postgresql.html) before upgrade to 12.1 and beyond. +## Application services client configuration + +Application services client database connection options can be found +in the [application services database client configuration](app_configuration.md) +documentation. + +## Omnibus GitLab-bundled PostgreSQL service configuration + +### Store PostgreSQL data in a different directory + +By default, Omnibus GitLab stores PostgreSQL-related data in the +`/var/opt/gitlab/postgresql` directory. To change it: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + postgresql['dir'] = '/path/to/postgresql' + ``` + +1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) + GitLab for the changes to take effect. + +Note that changing the setting affects the following items: + +- The actual PostgreSQL data, by default stored in `/var/opt/gitlab/postgresql/data`, + will now be stored in this directory in a `data` sub-directory. +- The Unix socket used to connect to PostgreSQL, by default located at + `/var/opt/gitlab/postgresql/.s.PGSQL.5432`, will be located in this directory. + This can be configured separately with the `postgresql['unix_socket_directory']` + configuration directive. +- The `HOME` directory of the `gitlab-psql` system will be set to this + directory. This can be configured separately with the `postgresql['home']` + configuration directive. + +### Enable PostgreSQL service to listen over TCP/IP + +The packaged PostgreSQL server can be configured to listen for TCP/IP +connections, with the caveat that some non-critical scripts expect UNIX sockets +and may misbehave. To enable it: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + postgresql['listen_address'] = '0.0.0.0' + postgresql['port'] = 5432 + + # Configure the username and MD5 username-password hash used to access the + # GitLab database + postgresql['sql_user'] = 'gitlab' + postgresql['sql_user_password'] = 'database_password' + + # Configure the list of CIDR address blocks allowed to connect after password + # authentication + postgresql['md5_auth_cidr_addresses'] = %w(10.200.0.1/24 10.300.0.1/24) + + # Configure the list of CIDR address blocks allowed to connect without + # authentication of any kind. Be very careful with this with. It is suggested + # to limit this to the loopback address of 17.0.0.1/24 or even 127.0.0.1/32. + postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/24) + ``` + +1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) + GitLab for the changes to take effect. + +### Configure SSL mode + +Omnibus GitLab automatically enables SSL on the PostgreSQL service, but will +accept both encrypted and unencrypted connections by default. This behavior can +be adjusted by changing the [SSL mode](https://www.postgresql.org/docs/9.6/libpq-ssl.html#LIBPQ-SSL-PROTECTION) +that the PostgreSQL service runs in. To set this: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + postgresql['db_sslmode'] = 'require' + ``` + +1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) + GitLab to apply the configuration changes. + +1. Restart PostgreSQL for the changes to take effect: + + ```shell + sudo gitlab-ctl restart postgresql + ``` + +If PostgreSQL fails to start, check the logs at `/var/log/gitlab/postgresql/current` +for more details. + +#### + ## Provide your own PostgreSQL instance **(CORE ONLY)** It is possible to use GitLab with an [external PostgreSQL service](external_database.md). -- GitLab From a013dd1983e122234a41e29adca073640044916b Mon Sep 17 00:00:00 2001 From: Wei-Meng Lee Date: Tue, 31 Mar 2020 13:34:48 +0000 Subject: [PATCH 5/8] Add password warning --- doc/administration/database/app_configuration.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/administration/database/app_configuration.md b/doc/administration/database/app_configuration.md index 239f6c856ec133..f65c0c192734a9 100644 --- a/doc/administration/database/app_configuration.md +++ b/doc/administration/database/app_configuration.md @@ -42,6 +42,10 @@ default value of `60000` (60 seconds) is used. gitlab_rails['db_password'] = 'database_password' ``` + NOTE: **Note:** + `/etc/gitlab/gitlab.rb` should have file permissions `0600` because it contains + plain-text passwords. + 1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) GitLab for the changes to take effect. -- GitLab From c2fc7e2ef4d00581ff0319e46b59f3114b9e6c68 Mon Sep 17 00:00:00 2001 From: Wei-Meng Lee Date: Tue, 31 Mar 2020 13:43:37 +0000 Subject: [PATCH 6/8] Even more updates --- doc/administration/database/index.md | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/doc/administration/database/index.md b/doc/administration/database/index.md index b004d4c4330603..3f76fc375bdf52 100644 --- a/doc/administration/database/index.md +++ b/doc/administration/database/index.md @@ -22,9 +22,22 @@ documentation. ### Store PostgreSQL data in a different directory +CAUTION: **Caution:** +This is an intrusive operation. It cannot be done without downtime on an +existing installation. + By default, Omnibus GitLab stores PostgreSQL-related data in the `/var/opt/gitlab/postgresql` directory. To change it: +1. Stop GitLab: + + ```shell + sudo gitlab-ctl stop + ``` + +1. If you have existing PostgreSQL data, copy them from the old location to the + new location. + 1. Edit `/etc/gitlab/gitlab.rb`: ```ruby @@ -34,6 +47,12 @@ By default, Omnibus GitLab stores PostgreSQL-related data in the 1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) GitLab for the changes to take effect. +1. Start GitLab: + + ```shell + sudo gitlab-ctl start + ``` + Note that changing the setting affects the following items: - The actual PostgreSQL data, by default stored in `/var/opt/gitlab/postgresql/data`, @@ -46,7 +65,7 @@ Note that changing the setting affects the following items: directory. This can be configured separately with the `postgresql['home']` configuration directive. -### Enable PostgreSQL service to listen over TCP/IP +### Allow PostgreSQL service to listen over TCP/IP The packaged PostgreSQL server can be configured to listen for TCP/IP connections, with the caveat that some non-critical scripts expect UNIX sockets @@ -76,7 +95,7 @@ and may misbehave. To enable it: 1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) GitLab for the changes to take effect. -### Configure SSL mode +### Configure PostgreSQL SSL mode Omnibus GitLab automatically enables SSL on the PostgreSQL service, but will accept both encrypted and unencrypted connections by default. This behavior can @@ -101,16 +120,13 @@ that the PostgreSQL service runs in. To set this: If PostgreSQL fails to start, check the logs at `/var/log/gitlab/postgresql/current` for more details. -#### - ## Provide your own PostgreSQL instance **(CORE ONLY)** It is possible to use GitLab with an [external PostgreSQL service](external_database.md). ## Standalone PostgreSQL server using Omnibus GitLab **(CORE ONLY)** -You can use the GitLab Omnibus package to easily -deploy the bundled PostgreSQL. +You can use the GitLab Omnibus package to easily deploy the bundled PostgreSQL. 1. SSH into the PostgreSQL server. 1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab -- GitLab From e00747212f24ba7d984a3178c177104c03a39c5c Mon Sep 17 00:00:00 2001 From: Wei-Meng Lee Date: Tue, 31 Mar 2020 21:52:33 +0800 Subject: [PATCH 7/8] Fix linting errors --- .../database/app_configuration.md | 4 ++-- .../database/external_database.md | 16 +++++++------- doc/administration/database/index.md | 22 +++++++++---------- 3 files changed, 20 insertions(+), 22 deletions(-) diff --git a/doc/administration/database/app_configuration.md b/doc/administration/database/app_configuration.md index f65c0c192734a9..2d3d35c720f675 100644 --- a/doc/administration/database/app_configuration.md +++ b/doc/administration/database/app_configuration.md @@ -51,7 +51,7 @@ default value of `60000` (60 seconds) is used. **For installations from source** -Follow the [Configure GitLab DB Settings](../install/installation.html#configure-gitlab-db-settings) section in the +Follow the [Configure GitLab DB Settings](../../install/installation.md#configure-gitlab-db-settings) section in the installation from source documentation. ### Verify server SSL certificate against CA bundle @@ -80,7 +80,7 @@ certificate against a CA bundle to prevent spoofing. article on AWS. 1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) - GitLab to apply the configuration changes. + GitLab to apply the configuration changes. 1. Restart PostgreSQL for the changes to take effect: diff --git a/doc/administration/database/external_database.md b/doc/administration/database/external_database.md index 04c46f37d8a0fe..e0f4c60d05f1df 100644 --- a/doc/administration/database/external_database.md +++ b/doc/administration/database/external_database.md @@ -16,7 +16,7 @@ separate from the GitLab Omnibus package. ### Requirements Your external PostgreSQL service needs to be set up according to the GitLab -[database requirements documentation](../../install/requirements.html#postgresql-requirements). +[database requirements documentation](../../install/requirements.md#postgresql-requirements). You will also need to set up a user (Omnibus GitLab uses `gitlab` by default) with a password of your choice. This user needs privileges to create the @@ -42,7 +42,7 @@ with a password of your choice. This user needs privileges to create the **For installations from source** -Follow the [Configure GitLab DB Settings](../install/installation.html#configure-gitlab-db-settings) section in the +Follow the [Configure GitLab DB Settings](../../install/installation.md#configure-gitlab-db-settings) section in the installation from source documentation. NOTE: **Note:** @@ -67,12 +67,12 @@ sudo gitlab-rake gitlab:setup **For installations from source** -Follow the [Initialize Database](../install/installation.html#initialize-database-and-activate-advanced-features) section in the +Follow the [Initialize Database](../../install/installation.md#initialize-database-and-activate-advanced-features) section in the installation from source documentation. ## Backup and restore using rake task -When using the [rake backup create and restore task](../../raketasks/backup_restore.md#create-a-backup-of-the-gitlab-system), +When using the [rake backup create and restore task](../../raketasks/backup_restore.md), GitLab will attempt to use the packaged `pg_dump` command to craete a database backup file and the packaged `psql` command to restore a backup. This will only work if they are the correct versions. To check the versions of the packaged @@ -86,7 +86,7 @@ work if they are the correct versions. To check the versions of the packaged If these versions are different from your external PostgreSQL service, you will need to install tools that match your database version and then follow the steps below. There are multiple ways to install PostgreSQL client tools. See -https://www.postgresql.org/download/ for options. +the [PostgreSQL downloads page](https://www.postgresql.org/download/) for options. Once the correct `psql` and `pg_dump` tools are available on your system, follow these steps, using the correct path to the location you installed the @@ -108,9 +108,9 @@ new tools: They should now be the same as your external PostgreSQL service. After this is done, ensure that the backup and restore tasks are using the -correct executables by running both the [backup](../../raketasks/backup_restore.md#create-a-backup-of-the-gitlab-system) -and [restore](../../raketasks/backup_restore.html#restore-a-previously-created-backup) -tasks. +correct executables by running both the +[backup](../../raketasks/backup_restore.md#creating-a-backup-of-the-gitlab-system) +and [restore](../../raketasks/backup_restore.md#restore) tasks. ## Upgrading the external PostgreSQL service diff --git a/doc/administration/database/index.md b/doc/administration/database/index.md index 3f76fc375bdf52..9052d788a4173e 100644 --- a/doc/administration/database/index.md +++ b/doc/administration/database/index.md @@ -8,8 +8,7 @@ GitLab supports only the use of PostgreSQL as its database. NOTE: **Note:** Prior to GitLab 12.1, MySQL was supported for Enterprise Edition licensed -instances. If you're using MySQL, you need to -[migrate to PostgreSQL](../../update/mysql_to_postgresql.html) +instances. If you're using MySQL, you need to [migrate to PostgreSQL](../../update/mysql_to_postgresql.md) before upgrade to 12.1 and beyond. ## Application services client configuration @@ -18,7 +17,7 @@ Application services client database connection options can be found in the [application services database client configuration](app_configuration.md) documentation. -## Omnibus GitLab-bundled PostgreSQL service configuration +## Omnibus-bundled PostgreSQL configuration ### Store PostgreSQL data in a different directory @@ -27,10 +26,10 @@ This is an intrusive operation. It cannot be done without downtime on an existing installation. By default, Omnibus GitLab stores PostgreSQL-related data in the -`/var/opt/gitlab/postgresql` directory. To change it: +`/var/opt/gitlab/postgresql` directory. To change it: + +1. Stop GitLab: -1. Stop GitLab: - ```shell sudo gitlab-ctl stop ``` @@ -45,7 +44,7 @@ By default, Omnibus GitLab stores PostgreSQL-related data in the ``` 1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) - GitLab for the changes to take effect. + GitLab for the changes to take effect. 1. Start GitLab: @@ -75,7 +74,7 @@ and may misbehave. To enable it: ```ruby postgresql['listen_address'] = '0.0.0.0' - postgresql['port'] = 5432 + postgresql['port'] = 5432 # Configure the username and MD5 username-password hash used to access the # GitLab database @@ -93,7 +92,7 @@ and may misbehave. To enable it: ``` 1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) - GitLab for the changes to take effect. + GitLab for the changes to take effect. ### Configure PostgreSQL SSL mode @@ -109,7 +108,7 @@ that the PostgreSQL service runs in. To set this: ``` 1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) - GitLab to apply the configuration changes. + GitLab to apply the configuration changes. 1. Restart PostgreSQL for the changes to take effect: @@ -180,8 +179,7 @@ You can use the GitLab Omnibus package to easily deploy the bundled PostgreSQL. 1. Note the PostgreSQL node's IP address or hostname, port, and plain text password. These will be necessary when configuring the GitLab application servers later. -1. [Enable monitoring](#enable-monitoring) +1. [Enable monitoring] Advanced configuration options are supported and can be added if needed. - -- GitLab From 8186903d773ecb5ae49f85404ec271ee5de295fa Mon Sep 17 00:00:00 2001 From: Wei-Meng Lee Date: Wed, 1 Apr 2020 01:36:24 +0800 Subject: [PATCH 8/8] Transfer more docs --- doc/administration/database/index.md | 317 +++++++++++++++--- .../database/standalone_database.md | 60 ++++ 2 files changed, 322 insertions(+), 55 deletions(-) create mode 100644 doc/administration/database/standalone_database.md diff --git a/doc/administration/database/index.md b/doc/administration/database/index.md index 9052d788a4173e..6ed895204f4c53 100644 --- a/doc/administration/database/index.md +++ b/doc/administration/database/index.md @@ -11,15 +11,34 @@ Prior to GitLab 12.1, MySQL was supported for Enterprise Edition licensed instances. If you're using MySQL, you need to [migrate to PostgreSQL](../../update/mysql_to_postgresql.md) before upgrade to 12.1 and beyond. -## Application services client configuration +This page documents administration tasks for the Omnibus-bundled PostgreSQL +service. For details on other aspects of GitLab database administration, please +read: -Application services client database connection options can be found -in the [application services database client configuration](app_configuration.md) -documentation. +- [Configuring the GitLab application services' database client](app_configuration.md) +- [Using an external database with GitLab](external_database.md) +- [Standalone PostgreSQL server using Omnibus GitLab](standalone_database.md) -## Omnibus-bundled PostgreSQL configuration +## Omnibus-bundled PostgreSQL -### Store PostgreSQL data in a different directory +### Connecting to the bundled PostgreSQL database + +If you need to connect to the bundled PostgreSQL database, you can +connect as the application user: + +```shell +sudo gitlab-rails dbconsole +``` + +or as a PostgreSQL superuser: + +```shell +sudo gitlab-psql -d gitlabhq_production +``` + +### Configuration + +#### Storing PostgreSQL data in a different directory CAUTION: **Caution:** This is an intrusive operation. It cannot be done without downtime on an @@ -64,7 +83,7 @@ Note that changing the setting affects the following items: directory. This can be configured separately with the `postgresql['home']` configuration directive. -### Allow PostgreSQL service to listen over TCP/IP +#### Allowing PostgreSQL service to listen over TCP/IP The packaged PostgreSQL server can be configured to listen for TCP/IP connections, with the caveat that some non-critical scripts expect UNIX sockets @@ -94,7 +113,7 @@ and may misbehave. To enable it: 1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) GitLab for the changes to take effect. -### Configure PostgreSQL SSL mode +#### Configuring PostgreSQL SSL mode Omnibus GitLab automatically enables SSL on the PostgreSQL service, but will accept both encrypted and unencrypted connections by default. This behavior can @@ -119,67 +138,255 @@ that the PostgreSQL service runs in. To set this: If PostgreSQL fails to start, check the logs at `/var/log/gitlab/postgresql/current` for more details. -## Provide your own PostgreSQL instance **(CORE ONLY)** +#### Configuring PostgreSQL to use a custom SSL certificate -It is possible to use GitLab with an [external PostgreSQL service](external_database.md). +Omnibus GitLab automatically generates a self-signed certificate and private key +and stores them in the `/var/opt/gitlab/postgresql/data` directory by default. -## Standalone PostgreSQL server using Omnibus GitLab **(CORE ONLY)** +If you'd prefer to use a CA-signed certificate, you'll need: -You can use the GitLab Omnibus package to easily deploy the bundled PostgreSQL. +1. The public SSL certificate for the database (`server.crt`). +1. The corresponding private key for the SSL certificate (`server.key`). +1. A root certificate bundle that validates the server's certificate + (`cacert.pem`). By default, Omnibus GitLab will use the embedded certificate + bundle in `/opt/gitlab/embedded/ssl/certs/cacert.pem`. -1. SSH into the PostgreSQL server. -1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab - package you want using **steps 1 and 2** from the GitLab downloads page. - - Do not complete any other steps on the download page. -1. Generate a password hash for PostgreSQL. This assumes you will use the default - username of `gitlab` (recommended). The command will request a password - and confirmation. Use the value that is output by this command in the next - step as the value of `POSTGRESQL_PASSWORD_HASH`. +For more details, see the [PostgreSQL documentation](https://www.postgresql.org/docs/9.6/ssl-tcp.html). - ```shell - sudo gitlab-ctl pg-password-md5 gitlab - ``` +Note that `server.crt` and `server.key` may be different from the default SSL +certificates used to access GitLab. For example, suppose the external hostname +of your database is `database.example.com`, and your external GitLab hostname +is `gitlab.example.com`. You will either need a wildcard certificate for +`*.example.com` or two different SSL certificates. -1. Edit `/etc/gitlab/gitlab.rb` and add the contents below, updating placeholder - values appropriately. +With these files in hand, enable SSL: - - `POSTGRESQL_PASSWORD_HASH` - The value output from the previous step - - `APPLICATION_SERVER_IP_BLOCKS` - A space delimited list of IP subnets or IP - addresses of the GitLab application servers that will connect to the - database. Example: `%w(123.123.123.123/32 123.123.123.234/32)` +1. Edit `/etc/gitlab/gitlab.rb`: ```ruby - # Disable all components except PostgreSQL - roles ['postgres_role'] - repmgr['enable'] = false - consul['enable'] = false - prometheus['enable'] = false - alertmanager['enable'] = false - pgbouncer_exporter['enable'] = false - redis_exporter['enable'] = false - gitlab_exporter['enable'] = false + postgresql['ssl_cert_file'] = '/custom/path/to/server.crt' + postgresql['ssl_key_file'] = '/custom/path/to/server.key' + postgresql['ssl_ca_file'] = '/custom/path/to/cacert.pem' + postgresql['internal_certificate'] = "-----BEGIN CERTIFICATE----- + ...base64-encoded certificate... + -----END CERTIFICATE----- + " + postgresql['internal_key'] = "-----BEGIN RSA PRIVATE KEY----- + ...base64-encoded private key... + -----END RSA PRIVATE KEY----- + " + ``` - postgresql['listen_address'] = '0.0.0.0' - postgresql['port'] = 5432 + Relative paths will be rooted from the PostgreSQL data directory + (`/var/opt/gitlab/postgresql/data` by default). - # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value - postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH' + NOTE: **Note:** + You must ensure that the `gitlab-psql` user can access the directory the + files are placed in and can read the private key. Omnibus will automatically + manage the permissions of the files for you. - # Replace XXX.XXX.XXX.XXX/YY with Network Address - # ???? - postgresql['trust_auth_cidr_addresses'] = %w(APPLICATION_SERVER_IP_BLOCKS) +1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) + GitLab to apply the configuration changes. - # Disable automatic database migrations - gitlab_rails['auto_migrate'] = false +1. Restart PostgreSQL for the changes to take effect: + + ```shell + sudo gitlab-ctl restart postgresql ``` - NOTE: **Note:** The role `postgres_role` was introduced with GitLab 10.3 +### Verifying that SSL is being used + +To check whether SSL is being used by clients, you can run: + +```shell +sudo gitlab-rails dbconsole +``` + +At startup, you should see a banner as the following: + +```shell +psql (9.6.5) +SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: on) +Type "help" for help. +``` + +To check whether clients are using SSL, you can issue this SQL query: + +```sql +SELECT * FROM pg_stat_ssl; +``` + +For example: + +```sql +gitlabhq_production=> SELECT * FROM pg_stat_ssl; + pid | ssl | version | cipher | bits | compression | clientdn +-------+-----+---------+-----------------------------+------+-------------+---------- + 47506 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | + 47509 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | + 47510 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | + 47527 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | + 47528 | f | | | | | + 47537 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | + 47560 | f | | | | | + 47561 | f | | | | | + 47563 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | + 47564 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | + 47565 | f | | | | | + 47569 | f | | | | | + 47570 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | + 47573 | f | | | | | + 47585 | f | | | | | + 47586 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | + 47618 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | + 47628 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | + 55812 | t | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 | 256 | t | +(19 rows) +``` + +Rows that have `t` listed under the `ssl` column are enabled. + +### Upgrading + +Omnibus GitLab will automatically update PostgreSQL to the +[default shipped version](https://docs.gitlab.com/omnibus/package-information/postgresql_versions.md) +during package upgrades unless specifically opted out. + +To opt out of automatic PostgreSQL upgrade during GitLab package upgrades, run: + +```shell +sudo touch /etc/gitlab/disable-postgresql-upgrade +``` + +If you want to manually upgrade the PostgreSQL service, you can follow these +instructions: + +**Note:** + +- Fully read this section before running any commands. +- Plan ahead as upgrade involves downtime. +- If you encounter any problems during upgrade, please raise an issue with a + full description in the [Omnibus GitLab issue tracker](https://gitlab.com/gitlab-org/omnibus-gitlab). + +Before upgrading, check the following: + +- You're currently running the latest version of GitLab and it is working. +- If you recently upgraded, make sure that a [GitLab reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) + ran successfully before you proceed. +- You will need to have sufficiency disk space for two copies of your database. + **Do not attempt to upgrade unless you have enough free space available.** + Check your database size using `sudo su -sh /var/opt/gitlab/postgresql/data` + (or your database path, if customized) and space available using `sudo df -h`. + If the parition where the database resides does not have enough space, you can + pass the argument `--tmp-dir $DIR` to the command. + +NOTE: **Note:** +The upgrade requires downtime as the database must be down while the upgrade is +being performed. The length of downtime depends on the size of your database. If +you would rather avoid downtime, it is possible to upgrade to a new database +using [Slony](https://www.slony.info/). Please see our [guide](../../update/upgrading_postgresql_using_slony.md) +on how to perform the upgrade. + +Once you have confirmed that the above checklist is satisfied, you can proceed. +To perform the upgrade, run the command: + +```shell +sudo gitlab-ctl pg-upgrade +``` + +NOTE: **Note:** +In GitLab 12.8 or later, you can pass the `-V 11` flag to opt in to upgrading to PostgreSQL 11. + +This command performs the following steps: + +1. Checks to ensure the database is in a known good state +1. Shuts down the existing database, any unnecessary services, and enables the + GitLab deploy page. +1. Changes the symlinks in `/opt/gitlab/embedded/bin/` for PostgreSQL to point + to the newer version of the database +1. Creates a new directory containing a new, empty database with a locale + matching the existing database +1. Uses the `pg_upgrade` tool to copy the data from the old database to the new + database +1. Moves the old database out of the way +1. Moves the new database to the expected location +1. Calls `sudo gitlab-ctl reconfigure` to do the required configuration changes, + and start the new database server. +1. Start the remaining services, and remove the deploy page. +1. If any errors are detected during this process, it should immediately revert + to the old version of the database. + +Once this step is complete, verify everything is working as expected. + +Once this step is complete, verify everything is working as expected. + +**Once you have verified that your GitLab instance is running correctly**, +you can clean up the old database files with: + +```shell +sudo rm -rf /var/opt/gitlab/postgresql/data. +sudo rm -f /var/opt/gitlab/postgresql-version.old +``` + +You can find details of PostgreSQL versions shipped with various GitLab versions +in [PostgreSQL versions shipped with Omnibus GitLab](https://docs.gitlab.com/omnibus/package-information/postgresql_versions.md). +The following section details their update policy. + +#### GitLab 12.8 and later + +**As of GitLab 12.8, PostgreSQL 9.6.17, 10.12, and 11.7 are shipped with +Omnibus GitLab.** + +Automatically during package upgrades (unless opted out) and when user manually +runs `gitlab-ctl pg-upgrade`, `omnibus-gitlab` will still be attempting to +upgrade the database only to 10.x, while 11.x will be available for users to +manually upgrade to. To manually update PostgreSQL to version 11.x , the `pg-upgrade` +command has to be passed with a version argument (`-V` or `--target-version`) + +```shell +sudo gitlab-ctl pg-upgrade -V 11 +``` + +NOTE: **Note:** +We **DO NOT** recommend updating to PostgreSQL 11.x on GitLab instances making use of +GitLab Geo for replication, as we have not yet completed PostgreSQL 11 testing with GitLab +Geo. We will be [completing this work](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4975) +in a future release. + +#### GitLab 12.0 and later + +**As of GitLab 12.0, PostgreSQL 9.6.11 and 10.7 are shipped with Omnibus +GitLab.** + +On upgrades, we will be automatically upgrading the database to 10.7 unless +specifically opted out as described above. + +#### GitLab 11.11 and later + +**As of GitLab 11.11, PostgreSQL 9.6.X and 10.7 are shipped with Omnibus +GitLab.** + +Fresh installs will be getting PostgreSQL 10.7 while GitLab package upgrades +will retain the existing version of PostgreSQL. Users can manually upgrade to +the 10.7 using the `pg-upgrade` command as mentioned above. + +### Downgrading + +On GitLab versions which ship multiple PostgreSQL versions, users can downgrade +an already upgraded PostgreSQL version to the earlier version using: + +```shell +sudo gitlab-ctl revert-pg-upgrade +``` + +This command also supports the `-V` flag to specify a target version for +scenarios where more than two PostgreSQL versions are shipped in the package. +For example, GitLab 12.8 ships with PostgreSQL 9.6.x, 10.x and 11.x. -1. [Reconfigure GitLab] for the changes to take effect. -1. Note the PostgreSQL node's IP address or hostname, port, and - plain text password. These will be necessary when configuring the GitLab - application servers later. -1. [Enable monitoring] +If the target version is not specified, it will use the version in +`/var/opt/gitlab/postgresql-version.old` if available. Otherwise, it falls back +to the default version shipped with GitLab. -Advanced configuration options are supported and can be added if -needed. +On other GitLab versions which ship only one PostgreSQL version, you can't +downgrade your PostgreSQL version. You must downgrade GitLab to an older version +to do this. diff --git a/doc/administration/database/standalone_database.md b/doc/administration/database/standalone_database.md new file mode 100644 index 00000000000000..3366cba2c75203 --- /dev/null +++ b/doc/administration/database/standalone_database.md @@ -0,0 +1,60 @@ +# Standalone PostgreSQL server using Omnibus GitLab **(CORE ONLY)** + +You can use the GitLab Omnibus package to easily deploy the bundled PostgreSQL. + +1. SSH into the PostgreSQL server. +1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab + package you want using **steps 1 and 2** from the GitLab downloads page. + - Do not complete any other steps on the download page. +1. Generate a password hash for PostgreSQL. This assumes you will use the default + username of `gitlab` (recommended). The command will request a password + and confirmation. Use the value that is output by this command in the next + step as the value of `POSTGRESQL_PASSWORD_HASH`. + + ```shell + sudo gitlab-ctl pg-password-md5 gitlab + ``` + +1. Edit `/etc/gitlab/gitlab.rb` and add the contents below, updating placeholder + values appropriately. + + - `POSTGRESQL_PASSWORD_HASH` - The value output from the previous step + - `APPLICATION_SERVER_IP_BLOCKS` - A space delimited list of IP subnets or IP + addresses of the GitLab application servers that will connect to the + database. Example: `%w(123.123.123.123/32 123.123.123.234/32)` + + ```ruby + # Disable all components except PostgreSQL + roles ['postgres_role'] + repmgr['enable'] = false + consul['enable'] = false + prometheus['enable'] = false + alertmanager['enable'] = false + pgbouncer_exporter['enable'] = false + redis_exporter['enable'] = false + gitlab_exporter['enable'] = false + + postgresql['listen_address'] = '0.0.0.0' + postgresql['port'] = 5432 + + # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value + postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH' + + # Replace XXX.XXX.XXX.XXX/YY with Network Address + # ???? + postgresql['trust_auth_cidr_addresses'] = %w(APPLICATION_SERVER_IP_BLOCKS) + + # Disable automatic database migrations + gitlab_rails['auto_migrate'] = false + ``` + + NOTE: **Note:** The role `postgres_role` was introduced with GitLab 10.3 + +1. [Reconfigure GitLab] for the changes to take effect. +1. Note the PostgreSQL node's IP address or hostname, port, and + plain text password. These will be necessary when configuring the GitLab + application servers later. +1. [Enable monitoring] + +Advanced configuration options are supported and can be added if +needed. -- GitLab