AIP-144

Repeated fields

Representing lists of data in an API is trickier than it often appears. Users often need to modify lists in place, and longer data series within a single resource pose a challenge for pagination.

Guidance

Resources may use repeated fields where appropriate.

message Book {
  option (google.api.resource) = {
    type: "library.googleapis.com/Book"
    pattern: "publishers/{publisher}/books/{book}"
  };

  string name = 1 [(google.api.field_behavior) = IDENTIFIER];

  repeated string authors = 2;
}
  • Repeated fields must use a plural field name.
    • If the English singular and plural words are identical ("moose", "info"), the dictionary word must be used rather than attempting to coin a new plural form.
  • Repeated fields should have an enforced upper bound that will not cause a single resource payload to become too large. A good rule of thumb is 100 elements.
    • If repeated data has the chance of being too large, the API should use a sub-resource instead.
  • Repeated fields must not represent the body of another resource inline. Instead, the message should provide the resource names of the associated resources.

Scalars and messages

Repeated fields should use a scalar type (such as string) if they are certain that additional data will not be needed in the future, as using a message type adds significant cognitive overhead and leads to more complicated code.

However, if additional data is likely to be needed in the future, repeated fields should use a message instead of a scalar proactively, to avoid parallel repeated fields.

Update strategies

A resource may use two strategies to enable updating a repeated field: direct update using the standard Update method, or custom Add and Remove methods.

A standard Update method has one key limitation: the user is only able to update the entire list. Field masks are unable to address individual entries in a repeated field. This means that the user must read the resource, make modifications to the repeated field value as needed, and send it back. This is fine for many situations, particularly when the repeated field is expected to have a small size (fewer than 10 or so) and race conditions are not an issue, or can be guarded against with ETags.

Note: Declarative-friendly resources must use the standard Update method, and not introduce Add and Remove methods. If declarative tools need to reason about particular relationships while ignoring others, consider using a subresource instead.

If atomic modifications are required, the API should define custom methods using the verbs Add and Remove:

Note: If both of these strategies are too restrictive, consider using a subresource instead.

rpc AddAuthor(AddAuthorRequest) returns (Book) {
  option (google.api.http) = {
    post: "/v1/{book=publishers/*/books/*}:addAuthor"
    body: "*"
  };
}

rpc RemoveAuthor(RemoveAuthorRequest) returns (Book) {
  option (google.api.http) = {
    post: "/v1/{book=publishers/*/books/*}:removeAuthor"
    body: "*"
  };
}
  • The data being added or removed should be a primitive (usually a string).
    • For more complex data structures with a primary key, the API should use a map with the Update method instead.
  • The RPC's name must begin with the word Add or Remove. The remainder of the RPC name should be the singular form of the field being added.
  • The request message must match the RPC name, with a Request suffix.
  • The response message should be the resource itself, unless there is useful context to provide in the response, in which case the response message must match the RPC name, with a Response suffix.
    • When the response is the resource itself, it should include the fully-populated resource.
  • The HTTP verb must be POST, as is usual for custom methods.
  • The HTTP URI must end with :add* or :remove*, where * is the snake-case singular name of the field being added or removed.
  • The request message field receiving the resource name should map to the URI path.
    • The HTTP variable should be the name of the resource (such as book) rather than name or parent.
    • That variable should be the only variable in the URI path.
  • The body clause in the google.api.http annotation should be "*".
  • If the data being added in an Add RPC is already present, the method must error with ALREADY_EXISTS.
  • If the data being removed in a Remove RPC is not present, the method must error with NOT_FOUND.

Request Message

message AddAuthorRequest {
  // The name of the book to add an author to.
  string book = 1 [
    (google.api.field_behavior) = REQUIRED,
    (google.api.resource_reference).type = "library.googleapis.com/Book"
  ];

  string author = 2 [(google.api.field_behavior) = REQUIRED];
}

message RemoveAuthorRequest {
  // The name of the book to remove an author from.
  string book = 1 [
    (google.api.field_behavior) = REQUIRED,
    (google.api.resource_reference).type = "library.googleapis.com/Book"
  ];

  string author = 2 [(google.api.field_behavior) = REQUIRED];
}
  • A resource field must be included. It should be the name of the resource (such as book) rather than name or parent.
  • A field for the value being added or removed must be included. It should be the singular name of the field.
  • The request message must not contain any other required fields, and should not contain other optional fields except those described in this or another AIP.

Changelog

  • 2022-06-02: Changed suffix descriptions to eliminate superfluous "-".
  • 2020-10-17: Recommended returning the resource itself in Add and Remove RPCs over separate response types.
  • 2020-10-17: Added guidance for Add and Remove RPCs and requests.