Added Import::Offline::Export model and create service
What does this MR do and why?
This MR adds a new model, Import::Offline::Export
and table import_offline_exports
. Each offline export has many BulkImports::Export
to define what exported relations belong to one "export". Because the Import::Offline::Export
is somewhat long with its bounded context, I've tried to establish a pattern of naming them offline_export
in model associations, factories, etc. similar to source_user
s instead of import_source_user
s.
This MR also adds Import::Offline::Export::CreateService
, a basic service to create an Import::Offline::Export
if the user has permissions to export all groups and projects provided to the service. The service does not currently kick off the offline export process, that will be in another MR.
Notable implementation decisions:
- The relationship between
Import::Offline:Export
andBulkImports::Configuration
is not in this MR to keep scope small. -
batched
was not added as an attribute onImport::Offline:Export
despite being listed on the architecture design document. In direct transfer, all exports are batched if the source instance version is 16.2 or above, so all offline transfer exports will be batched as well to maintain consistency and stability. -
export_prefix
was not added as an attribute onImport::Offline:Export
despite being listed on the architecture design document.export_prefix
is effectively analogous to the import/export file download name, making it more of a configuration detail than an export option. It will also be needed in the offline transfer export and import process, so it didn't make much sense to add it toImport::Offline::Export
for now. This may change as development continues. -
source_hostname
was added toImport::Offline:Export
despite being onBulkImports::Configuration
in the architecture design document because it's an export option provided by the user.BulkImports::Configuration
s are also deleted 24 hours after a direct transfer completes, and it would make sense to continue that pattern for offline transfer. However, our users should be able to see whatsource_hostname
they entered on export for much longer than a day so that they can communicate what their contributors should expect to see when accepting a reassignment request. -
source_hostname
validation is currently quite strict and does not allow users to enter URLs with hostsgithub.com
,bitbucket.org
, orgitea.com
to help avoid particularly lazy attempts at malicious reassignment. It doesn't actually prevent a user from changing thesource_hostname
in export files if they're motivated enough to do so, however. If we find that these validations are too strict, it'll be easier to loosen validations in the future than make them more strict. - Within the rails app, direct transfer export and import/export use the term
portable
to refer to a group or project, and direct transfer imports use the termentity
to refer toBulkImports::Entity
objects. However, from the user's perspective,entity
refers to groups and projects in general (e.g. direct transfer API docs). I tried to maintain that pattern sinceportable
probably doesn't make a lot of sense to a user, and anentity
in direct transfer means something specific.
References
- Offline transfer architecture design document: https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/offline_direct_transfer_migrations
- Related to #538941
How to set up and validate locally
This MR does not implement any real functionality yet. To test locally,
- Run database migrations
- Enable
:offline_transfer_exports
feature flag - Run rspec tests, or open a rails console and create an
Import::Offline::Export
usingImport::Offline::Exports::CreateService
:
current_user = User.first # whatever user you want
source_hostname = 'https://offline-environment-gitlab.example.com' # Does not need to match the actual instance address
portable_params = [
{ type: 'group', full_path: 'gnuwget' },
{ type: 'project', full_path: 'toolbox/gitlab-smoke-tests' }
]
result = Import::Offline::Exports::CreateService.new(current_user, source_hostname, portable_params).execute
# result should be success and payload should be the new Import::Offline::Export
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #538941