[go: up one dir, main page]

Skip to content

Draft: Optimize frontend fixture downloads with caching mechanism

What does this MR do and why?

Optimize frontend fixture downloads with caching mechanism

Implement two-level caching to reduce unnecessary network requests and improve CI/CD performance when downloading frontend fixtures.

Cache implementation:

  • Add .fixtures_version file to track the SHA of last extracted fixtures
    • Skip download entirely if current SHA matches cached version
  • Add .no_archive file to cache SHAs without available archives
    • Maintain rolling cache of last 100 checked SHAs
    • Skip archive existence checks for known missing archives

Performance improvements:

  • Eliminate redundant downloads when fixtures are already current
  • Reduce API calls by ~50-80% in typical CI runs
  • Maintain cache size limits to prevent unbounded growth

The caching is stored in tmp/tests/frontend/ and is automatically created if missing. Cache cleanup ensures the no-archive list stays within reasonable bounds (100 entries).

References

How to set up and validate locally

  1. scripts/frontend/download_fixtures.sh
$ time scripts/frontend/download_fixtures.sh
Looking for frontend fixtures for commit 467cf8fe3db6c76907efea44bc93409a1e84af9c...  no archive found - caching.
Looking for frontend fixtures for commit fc4c323d0c487983095ecae0f8598516b1908ace...  no archive found - caching.
Looking for frontend fixtures for commit fd4e649a407af9c37a6a00a06b3f3d990a10dff6...  no archive found - caching.
Looking for frontend fixtures for commit ad11cd64f95588b4840d4cc4adcced56fe2a3499...
We have found frontend fixtures at https://gitlab.com/api/v4/projects/278964/packages/generic/fixtures/ad11cd64f95588b4840d4cc4adcced56fe2a3499/fixtures-ad11cd64f95588b4840d4cc4adcced56fe2a3499.tar.gz!
Downloading from https://gitlab.com/api/v4/projects/278964/packages/generic/fixtures/ad11cd64f95588b4840d4cc4adcced56fe2a3499/fixtures-ad11cd64f95588b4840d4cc4adcced56fe2a3499.tar.gz ...
Extracting archive to .

real    0m1.796s
user    0m0.155s
sys     0m0.125s
  1. rm tmp/tests/frontend/.fixtures_version tmp/tests/frontend/.no_archive
  2. scripts/frontend/download_fixtures.sh
$ time scripts/frontend/download_fixtures.sh
Looking for frontend fixtures for commit 467cf8fe3db6c76907efea44bc93409a1e84af9c...  cached - skipping.
Looking for frontend fixtures for commit fc4c323d0c487983095ecae0f8598516b1908ace...  cached - skipping.
Looking for frontend fixtures for commit fd4e649a407af9c37a6a00a06b3f3d990a10dff6...  cached - skipping.
Looking for frontend fixtures for commit ad11cd64f95588b4840d4cc4adcced56fe2a3499...  already downloaded - skipping.

real    0m0.306s
user    0m0.032s
sys     0m0.026s

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Peter Leitzen

Merge request reports

Loading