Draft: Save CI Job Definitions when pipeline creation

What does this MR do and why?

This MR implements the foundation for saving CI Job Definitions during pipeline creation as part of the broader CI data normalization initiative.

Quick summary:

• Ci::JobDefinition model to store immutable, deduplicated job configuration data
• Ci::JobDefinitionInstance model to link jobs to their definitions
• Jobs with identical configuration share the same JobDefinition record
• There is optimistic approach with retry logic for concurrent pipeline creation
• Changes are behind the write_to_new_ci_destinations feature flag

References

• Related to Add logic to populate new destinations when pip... (#551860 - closed)
• Inspired by Draft: PoC: split build metadata into `job_prot... (!193948 - closed)
• Part of Reduce growth rate of ci_builds_metadata partit... (&13886)

How to set up and validate locally

1. Enable the feature flag:

   Feature.enable(:write_to_new_ci_destinations)

2. Create a pipeline with jobs that have different configurations:

   test1:
     script: echo "test 1"
     variables:
       VAR1: value1
   
   test2:
     script: echo "test 2" 
     interruptible: true
   
   test3: # Same as test1
     script: echo "test 1"
     variables:
       VAR1: value1

3. Verify job definitions are created:

   pipeline = Ci::Pipeline.last
   pipeline.processables.each do |job|
     puts "#{job.name}: JobDefinition ID #{job.job_definition&.id}"
   end; nil

4. Expected: test1 and test3 should share the same JobDefinition ID

   test1: JobDefinition ID 4
   test2: JobDefinition ID 5
   test3: JobDefinition ID 4

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.