NGINX Ingress: Configure proxy request buffering for workhorse

What does this MR do?

GitLab workhorse needs customizable proxy_request_buffering settigs on a path level. This can be achieved using server snippets or by utilizing a customized NGINX template.

While the first option is more flexible, it requires additional NGINX configuration which is not recommended because of security concerns. Using a custom template allows us pass the necessary NGINX configuration without these concerns.

Reviewer notes

• The first commit is empty and has only the expected (squash) commit message.
• The second commit imports default NGINX configuration without changes.
• All functional and documentation changes are in the following commits.

Impact

1. This allows us to unblock several issues related to Git over SSH and should improve performance for large uploads and streaming requests.
2. This replicas known and tested Omnibus behavior: https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/e9dfceb9b27928fadf4154bea35ecd56575ec794/files/gitlab-cookbooks/gitlab/templates/default/nginx-gitlab-http.conf.erb#L183.
3. We will need to update the NGINX template when we update the controller image.

Related issues and MRs

• Attempt to achieve this using snippets: !4520 (closed)
• Inventory documenting our current NGINX proxy request buffering per component/traffic type: !4512
• Relates Research: Determine and Document impacts of dis... (#6034)
• Closes proxy_request_buffering should be disabled for ... (#2262)

Author checklist

For general guidance, please follow our Contributing guide.

Required

For anything in this list which will not be completed, please provide a reason in the MR discussion.

• Merge Request Title and Description are up to date, accurate, and descriptive.
• MR targeting the appropriate branch.
• MR has a green pipeline.
• Documentation created/updated.
• Tests added/updated, and test plan for scenarios not covered by automated tests.
  • This is tested best during executing. A syntax error in the NGINX template would cause our e2e tests to fail.
• Equivalent MR/issue for omnibus-gitlab opened.
  • Omnibus implements this behavior in https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/e9dfceb9b27928fadf4154bea35ecd56575ec794/files/gitlab-cookbooks/gitlab/attributes/default.rb#L927 and https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/e9dfceb9b27928fadf4154bea35ecd56575ec794/files/gitlab-cookbooks/gitlab/templates/default/nginx-gitlab-http.conf.erb#L183

Reviewers checklist

• MR has a green pipeline on https://gitlab.com/gitlab-org/charts/gitlab.
• Consider downstream impact to the Operator, as per evaluating impact from changes to GitLab chart.