Draft: Decouple artifact and metadata upload
What does this MR do and why?
Overview
This MR addresses timeout issues experienced by GitLab Runner when uploading large artifacts to the coordinator.
Implementation
The following changes have been implemented to resolve this issue:
- Decouple artifact and metadata upload: Workhorse now handles artifact uploads independently and only processes metadata upon request
- Delegate metadata processing: After successful artifact upload, Sidekiq takes responsibility for metadata generation and upload
- Improve response time: Workhorse returns a response to GitLab Runner immediately after artifact upload completion, without waiting for metadata processing
Benefits
This approach eliminates timeout issues by reducing the synchronous processing time and moving metadata handling to background processing.
References
Screenshots or screen recordings
| Before | After |
|---|---|
How to set up and validate locally
MR acceptance checklist
close gitlab-runner#37220
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.