[Feature flag] Enable rollout of non-repeating catfile cache keys

What

Enable the catfile_cache_nonrepeating feature flag.

Owners

• Team: Gitaly
• Most appropriate slack channel to reach out to: #g_gitaly
• Best individual to reach out to: @proglottis

Expectations

What release does this feature occur in first?

%17.10

What are we expecting to happen?

• Reduction in Internal error: `file already closed` (#6571 - closed)
• Possible change in catfile cache hit/miss ratio as keys will no longer rotate per-hour.

What might happen if this goes wrong?

• Build-up of catfile processes
• Excess cache misses

What can we monitor to detect problems with this?

• Metrics https://dashboards.gitlab.net/d/lov2nqd7k/gitaly-catfile-statistics
• Error logs as per Internal error: `file already closed` (#6571 - closed)

Roll Out Steps

• Enable on staging
  • Is the required code deployed on staging? (howto)
  • Enable on staging (howto)
  • Add featureflagstaging to this issue (howto)
  • Test on staging (howto)
  • Verify the feature flag was used by checking Prometheus metric gitaly_feature_flag_checks_total
• Enable on production
  • Is the required code deployed on production? (howto)
  • Progressively enable in production (howto)
  • Add featureflagproduction to this issue
  • Verify the feature flag was used by checking Prometheus metric gitaly_feature_flag_checks_total
• Create subsequent issues
  • To default enable the feature flag (optional, only required if backwards-compatibility concerns exist)
    • Create issue using the Feature Flag Default Enable template.
    • Set milestone to current+1 release
  • To Remove feature flag
    • Create issue using the Feature Flag Removal template.
    • Set milestone to current+1 (+2 if we created an issue to default enable the flag).

Please refer to the documentation of feature flags for further information.