[go: up one dir, main page]

Skip to content

task-runner backup-utility : parallel compression

Hi

Can we please change the backup-utility to use parallel compression? Currently it creates .tar.gz files (of mostly nearly compressible binary data, like registry images, of which I have roughly 500GB.) and gzip runs single threaded.

Possible alternatives are pbzip, xz, etc.
Eg, xz (LZMA2) with a low compression level provides good speeds and compression ratio.

Perhaps an option to not compress the registry would be useful as well.

https://gitlab.com/gitlab-org/build/CNG/blob/master/gitlab-task-runner/scripts/bin/backup-utility

Edited by Jan Smets