JMAP Conditional Set

Introduction The Foo/set method defined in is the mechanism by which clients create, update, and destroy objects. It provides one form of concurrency control: the optional "ifInState" argument, which aborts the entire method if the account's state string for that data type does not match the value the client supplies. This guard is coarse. The state string changes whenever any object of that type changes, by any client. In a busy account — or one that receives server-initiated changes, such as the arrival of new mail — the state can change between a client's read and its write for reasons entirely unrelated to the object the client is modifying. Using "ifInState" to protect a single update therefore leads to frequent spurious rejections and retries. Clients commonly need a narrower guarantee: "apply this change only if the specific object I read still holds the values I depend on". This is the same need met by the HTTP "If-Match" precondition () and the entity-tag model of WebDAV (): a conditional write scoped to a single resource. This document defines a generic, per-object conditional mechanism for Foo/set. It introduces no new properties and no new version tokens. Instead it reuses the PatchObject already defined by for updates: a client states its precondition as a PatchObject describing the values it expects the object to currently hold, and the server performs the change only if applying that patch would change nothing. Because the mechanism is defined entirely in terms of an object's properties and the existing PatchObject semantics, it applies uniformly to every data type that implements Foo/set, with no per-type additions. A second, related need is atomicity. processes the creations, updates, and destroys within a Foo/set independently: some may succeed while others fail, and the method gives no guarantee that a set of related changes takes effect together. The only all-or-nothing control Core offers is "ifInState", which — like the concurrency problem above — is scoped to the entire object type rather than to the changes the client actually requested. This document therefore also defines an optional "atomic" argument that commits a Foo/set as a single unit, evaluated against the state that results from applying all of its changes together (see ).

Conventions Used in This Document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. The terms "Id", "PatchObject", "SetError", and "Foo/set" are defined in . A "pointer" is a JSON Pointer as used within a PatchObject (see , Section 5.3).

Addition to the Capabilities Object

urn:ietf:params:jmap:conditional The presence of the "urn:ietf:params:jmap:conditional" property in the "capabilities" object of the JMAP Session resource indicates support for the conditional Foo/set behaviour defined in this document. The value of this property MUST be an empty object. A client indicates that it wishes to use this extension by including the capability URI in the "using" array of a request. When a server that advertises this capability receives a request whose "using" array includes this URI, it MUST honour the "ifUnchangedBy" argument () on every Foo/set method it implements. It MUST also honour the "atomic" argument (): either applying the method atomically, or, if it cannot, rejecting it with a "cannotApplyAtomically" error (). A server MUST NOT apply the changes of a method requested with "atomic" only in part. The capability is generic: it does not depend on any particular data type, and its presence applies to all types for which the server implements Foo/set.

Capability Example

Conditional Foo/set This document adds the following arguments to the standard Foo/set method (, Section 5.3): ifUnchangedBy: "Id[PatchObject|null]" (default: an empty object) A map of object id to a PatchObject expressing a precondition on that object. For each entry, the precondition is satisfied if and only if applying the given PatchObject to the current server-side object would leave the object unchanged. Equivalently: for every pointer in the PatchObject, the object's current value at that pointer MUST equal the value given in the PatchObject. A JSON "null" value matches a property that is not present (it asserts that removing the property would be a no-op). Comparison uses the same representation the server would return for that property from Foo/get. As in an update (, Section 5.3), a PatchObject with a pointer whose parent does not exist in the current object cannot be applied. Such a precondition is never satisfied, whatever the value, including "null": for example "participants/p1/name" cannot be satisfied by an object with no "participants" property. The server rejects the method as for an invalid pointer (rule 4 of ). A "null" value instead of a PatchObject asserts that the object does not exist: the precondition is satisfied if and only if no object with that id exists or is visible to the client, for example because it has been destroyed. An empty PatchObject ("{}") is valid and asserts only that the object exists: the precondition is satisfied if and only if an object with that id exists and is visible to the client. atomic: "Boolean" (default: false) If true, the Foo/set is applied as a single unit: the server MUST either commit every creation, update, and destroy the method requests, or make no change at all. See .

Evaluation Rules Preconditions are evaluated against the state of each object as it exists at the start of the method, before any create, update, or destroy in the same method is applied. Each key in "ifUnchangedBy" is the id of an object of the same type as the method (the "Foo" of "Foo/set"). The object need not be one the method changes. A client MAY set a precondition on an object the method does not change, for example its parent. A PatchObject precondition is not satisfied if its object does not exist, or is not visible to the client; a "null" precondition is not satisfied if it does. The server then handles it as in . A creation id (an id prefixed with "#") MAY be a key. It references an object created by an earlier method call in the same request (, Section 5.3), and stands for the id that creation produced. If no object of this type has that id, because the creation failed, created an object of a different type, or its object has since been destroyed, the precondition is evaluated as for an object that does not exist. This lets a client require that an object it created earlier in the request has not changed since then, or that it has since been destroyed. Unlike the PatchObject supplied in the "update" argument, which may only reference properties the client is permitted to set, the PatchObject in "ifUnchangedBy" MAY reference any property of the object, including server-set and otherwise read-only properties (for example a content identifier, a size, or a server-maintained change timestamp). This allows a client to condition a change on properties it cannot itself modify. If a pointer in an "ifUnchangedBy" PatchObject is not a valid pointer for the object's type, or the PatchObject is otherwise malformed, the server MUST reject the whole method, with either an "invalidArguments" error or a "stateMismatch" error as for an unsatisfied precondition. A server need not distinguish an invalid pointer from one that does not apply cleanly to the current object. The "ifUnchangedBy" and "ifInState" arguments are both method-level guards and compose. If both are supplied, the server checks "ifInState" first and then the "ifUnchangedBy" preconditions; if either does not hold, the method is rejected with a "stateMismatch" error and no change is made.

Effect If every precondition in "ifUnchangedBy" is satisfied, the method proceeds exactly as it would without this extension. If any precondition is not satisfied, the server MUST make no change — no creation, update, or destroy in the method takes effect — and MUST reject the whole method with a method-level "stateMismatch" error (). The state string for the type MUST be unchanged. This is all-or-nothing at the method level, by design: a single Foo/set is the unit of conditional application. A client that wants changes to succeed or fail independently of one another places them in separate method calls within the same request, each with its own "ifUnchangedBy".

The "stateMismatch" error A failed "ifUnchangedBy" precondition is reported using the method-level "stateMismatch" error already defined by for a failed "ifInState" guard. It is returned as an "error" response in place of the Foo/set response, so no create, update, or destroy result is returned and the client knows the method made no change. The error discloses no object data: a client that needs the current values fetches them with Foo/get or Foo/changes before deciding how to proceed. A server MAY include a "description" for debugging, and MAY include: failed: "Id[]" The ids of the objects whose preconditions were not satisfied. This lets a client re-fetch only what it needs, without disclosing any values. A server that does not wish to distinguish which precondition failed MAY omit this property.

Atomic Foo/set When the "atomic" argument is true, the server MUST apply the Foo/set as a single unit: either every creation, update, and destroy the method requests takes effect, or none of them does.

Evaluation The server evaluates the method as though all of its changes were applied together. Any constraint that or a data-type extension evaluates on the resulting object set — for example a uniqueness constraint on a name within a parent — is checked once, against the state produced by applying all the changes, not against any intermediate state. This guarantees that a set of changes that is valid only as a whole can succeed. For example, exchanging two names — renaming object A to the name currently held by B and B to the name currently held by A — collides if applied one change at a time, but the combined result is conflict-free. , Section 5.3 already permits a server to accept such a change when the final state is valid, but also permits it to process the changes sequentially and reject one of them, possibly after committing the other. With "atomic", the client can rely on the server evaluating the final state, and on the changes succeeding or failing together (see ).

Failure If any part of an atomic method fails — an invalid patch, a name collision, a permission error, a violated constraint, or any other SetError that would place an id in "notCreated", "notUpdated", or "notDestroyed" — the server MUST make no change and MUST reject the whole method with a method-level "atomicFailure" error. The state string for the type MUST be unchanged. A failed "ifUnchangedBy" precondition is reported as a "stateMismatch" error () whether or not "atomic" is set; preconditions are already all-or-nothing at the method level. The "atomicFailure" error is specific to the method's own creations, updates, and destroys not being committable together. The "atomicFailure" error object MAY carry "notCreated", "notUpdated", and "notDestroyed" maps — of the same form as a Foo/set response — identifying the ids whose processing failed and the SetError for each. Ids that would have succeeded are omitted; they were not committed, because the method as a whole was rejected. A server MAY report only the first failure it encounters.

Servers That Cannot Apply Atomically A server that advertises this capability but cannot apply a given method atomically — either at all, or because this particular method is beyond what it can commit as a unit — MUST reject the method with a method-level "cannotApplyAtomically" error and make no change. It MUST NOT fall back to applying the changes non-atomically. A client that receives "cannotApplyAtomically" MAY retry without "atomic" if it can tolerate partial application, or surface the failure otherwise. A client that does not recognise the error treats it as "serverFail" (): since a rejected atomic method makes no change, this fallback is safe, if less informative.

Examples

Force-with-lease on file content A client last synchronised a file node "f42" with content blob "G_old". It has since produced new content "G_new", and wishes to replace the content only if no other client has changed it in the meantime. It conditions on the server-set "blobId" ():

If "f42" still references "G_old", the update succeeds. If another client has already replaced the content (so "blobId" is now, say, "G_other"), the precondition fails and the server makes no change, returning a method-level "stateMismatch" error:

The client fetches "f42", resolves the conflict (for instance by preserving its own version as a separate file), and retries.

Conditional destroy Destroy a message only if it has not been read on another device. In , an unread message has no "$seen" keyword, so the client asserts that "keywords/$seen" is absent:

If the message has since been marked "$seen", the precondition fails, the whole method is rejected with a "stateMismatch" error, and "M7" is not deleted.

A condition that ignores unrelated changes Because the precondition references only the pointers the client supplies, concurrent changes to other properties of the same object do not cause it to fail. Here a client removes a grantee's access, but only if that grantee currently holds exactly the rights the client saw:

The change applies only if Bob currently has "mayRead" of "true". A concurrent change to an unrelated property of "d3" (its name, or another grantee's rights) does not trigger a "stateMismatch".

Conditioning on an object the method does not change A precondition may name an object the method is not creating, updating, or destroying. Here a client updates the content of file "f42", but only if it is still in the parent directory the client last saw ("d3"), so the write does not land in an unexpected place after a concurrent move:

If either "f42" has been reparented or "d3" has been renamed, the method is rejected with a "stateMismatch" error and the content is not written, even though "d3" itself is not being changed.

Conditioning on an object having been destroyed A "null" precondition requires that an object no longer exists. Here a client renames "t" to "report.txt", but only if the file "r" that used to hold that name has already been destroyed, by this client or another:

If "r" still exists, the method is rejected with a "stateMismatch" error and "t" keeps its name.

Conditioning on an object still existing An empty PatchObject requires only that an object still exists, whatever its current values. Here a client has generated a thumbnail of the file "f42" and uploads it as a new file, but only if "f42" has not been destroyed in the meantime, so that no orphaned thumbnail is left behind:

If "f42" no longer exists, the method is rejected with a "stateMismatch" error and no thumbnail is created. Changes to "f42" that leave it in place, such as a rename, do not cause the precondition to fail.

Whole-object compare-and-swap A data type that maintains a server-set change token can be used to require that nothing at all has changed. For example, a type with a server-maintained "changed" timestamp that is updated on every modification:

Because the server updates "changed" on any modification to the object, this asserts that the entire object is unchanged since the client last read it, recovering the coarse "If-Match" behaviour as a special case of the general mechanism.

Atomic exchange of two names Two file nodes, "a" (named "current.txt") and "b" (named "previous.txt"), are to swap names. Applied one at a time, either rename collides with the name the other still holds; applied together, the result is conflict-free. A server conforming only to may accept this swap, but is not required to. With "atomic" set, the rule that sibling names are unique is evaluated only against the resulting state:

Both renames take effect, or — if either is rejected for any reason — neither does, and the server returns an "atomicFailure" error identifying the offending id.

Atomic replace of one file by another A common pattern in POSIX applications is to write new content to a temporary file and then rename it over the original, so that other readers see either the complete old file or the complete new one, never a partial write. Here "t" (named "report.txt.tmp") holds the new content, and "r" (named "report.txt", with content "G_old") is the file to be replaced:

Without "atomic", a server may process each change sequentially against the current state (, Section 5.3), so the rename of "t" could be rejected because "r" still holds the name, while the destroy of "r" succeeds, leaving neither file named "report.txt". With "atomic", the sibling-name uniqueness rule is evaluated only against the resulting state, in which "r" no longer exists, and the two changes succeed or fail together. The "ifUnchangedBy" precondition additionally ensures that "r" is replaced only if no other client has changed its content since this client read it; if it has, the method is rejected with a "stateMismatch" error and neither file is changed. A client that wants the unconditional behaviour of POSIX rename(2) omits the precondition.

Implementation Considerations

Evaluating Preconditions A server can implement "ifUnchangedBy" generically, without code specific to any data type. For each entry, the server reads the current object in the representation it would return from Foo/get, applies the PatchObject to a copy of it using the same logic it uses for the "update" argument, and compares the result with the original. Any difference means the precondition is not satisfied. Every data type that supports Foo/set already has both a Foo/get representation and PatchObject application, so no new per-type machinery is required. If the PatchObject does not apply cleanly, the precondition is not satisfied, and the server may report it either way rule 4 of allows. A server MAY instead compare the referenced values directly, or use any cheaper method, provided the outcome is the same. The precondition check is only meaningful if no other change to the referenced objects can occur between evaluating the preconditions and applying the method's changes. A server therefore evaluates the preconditions and applies the changes within the same lock or transaction.

Applying Methods Atomically A server backed by a transactional store can implement "atomic" by performing the whole Foo/set within a single transaction, deferring constraint checks such as name uniqueness until all of the changes have been applied, and rolling the transaction back if any part fails. A server that cannot do this for some or all of its data types still supports this extension by returning "cannotApplyAtomically" () for those methods.

Implementation Status EDITOR: please remove this section before publication. This section records the status of known implementations of the protocol defined by this specification at the time of posting of this Internet-Draft, and is based on a proposal described in . The description of implementations in this section is intended to assist the IETF in its decision processes in progressing drafts to RFCs. Please note that the listing of any individual implementation here does not imply endorsement by the IETF. Furthermore, no effort has been spent to verify the information presented here that was supplied by IETF contributors. This is not intended as, and must not be construed to be, a catalog of available implementations or their features. Readers are advised to note that other implementations may exist.

Cyrus IMAP Organization: Cyrus IMAP project Description: "ifUnchangedBy" and "atomic" for all JMAP data types implemented by Cyrus. "cannotApplyAtomically" is returned only for operations that cannot be undone: immediate sends, and destroys that would remove data at once under the server's configured delete or expunge mode. Level of maturity: complete, under review (https://github.com/cyrusimap/cyrus-imapd/pull/6104) Licensing: BSD-style Contact: Bron Gondwana brong@fastmailteam.com

Fastmail Organization: Fastmail Description: "ifUnchangedBy" in Fastmail's internal JMAP server Level of maturity: in development Licensing: proprietary Contact: Bron Gondwana brong@fastmailteam.com

IANA Considerations

JMAP Capability Registration for "conditional" IANA is requested to register the "conditional" JMAP Capability as follows, in the "JMAP Capabilities" registry established by :

Capability Name:: urn:ietf:params:jmap:conditional
Specification document:: this document
Intended use:: common
Change Controller:: IETF
Security and privacy considerations:: this document,

JMAP Error Code Registration for "atomicFailure" IANA is requested to register the "atomicFailure" JMAP Error Code in the "JMAP Error Codes" registry established by :

JMAP Error Code:: atomicFailure
Intended use:: common
Change Controller:: IETF
Description:: A Foo/set requested with "atomic" set to true could not commit all of its changes together; no change was made. Returned as a method-level error.
Reference:: this document

JMAP Error Code Registration for "cannotApplyAtomically" IANA is requested to register the "cannotApplyAtomically" JMAP Error Code in the "JMAP Error Codes" registry established by :

JMAP Error Code:: cannotApplyAtomically
Intended use:: common
Change Controller:: IETF
Description:: A Foo/set was requested with "atomic" set to true, but the server cannot apply this method atomically; no change was made. Returned as a method-level error.
Reference:: this document

Security Considerations

Information Disclosure The conditional mechanism returns no object data. A failed precondition yields only a "stateMismatch" error, optionally naming the ids whose preconditions did not hold, but never their values — nothing beyond what the client could already obtain by reading the objects with Foo/get. A precondition does not bypass access control. A server MUST require the same permissions to read the properties referenced in an "ifUnchangedBy" PatchObject as it would to return them from Foo/get. In particular, a server MUST NOT evaluate a precondition on a property the requesting client is not permitted to read; such a condition MUST cause the method to fail with a "forbidden" error, so that the mechanism cannot be used as an oracle to probe the values of properties the client cannot otherwise see. This differs deliberately from a precondition on an object that does not exist or is not visible to the client, which is reported as "stateMismatch" (). For a client, an invisible object is indistinguishable from one that does not exist, and reporting it differently would disclose its existence. A property the client may not read, by contrast, belongs to an object the client can see; the restriction depends only on the client's rights, not on the property's value, so "forbidden" discloses nothing about the value and tells the client that the request cannot succeed however it is retried.

Atomic Methods and Resource Consumption Applying a method atomically may require the server to hold a transaction, or equivalent locks, across all of its changes, and so a very large atomic Foo/set can hold resources longer than the same changes made independently. The limits a server already applies to the number of objects in a single Foo/set () continue to apply, and a server MAY reject an atomic method it considers too large rather than attempt it.

Resource Consumption Evaluating a precondition is comparable in cost to validating an update PatchObject and imposes no significant additional load.

Changes EDITOR: please remove this section before publication. The source of this document exists on github at: https://github.com/brong/draft-gondwana-jmap-conditional/ draft-ietf-jmap-conditional-02 Clarified that a precondition with a pointer whose parent does not exist in the current object is never satisfied, whatever its value. Allowed a server to reject such a precondition, an invalid pointer, or a malformed PatchObject with either "invalidArguments" or "stateMismatch". A creation id stands for the id it produced; if no object of the method's type has that id, the precondition is evaluated as for an object that does not exist, rather than rejecting the method. Allowed "null" in place of a PatchObject, asserting that the object does not exist. Clarified that an empty PatchObject is valid and asserts only that the object exists, and added an example. Updated the Cyrus IMAP implementation status: "atomic" is now implemented. draft-ietf-jmap-conditional-01 Moved the examples into their own section; they were previously nested under the "Atomic Foo/set" section. Added an example of atomically replacing one file with another (the POSIX write-then-rename pattern). Added Implementation Considerations, describing a generic way to evaluate preconditions and to apply methods atomically. Added an Implementation Status section. Explained why a precondition on an unreadable property yields "forbidden" while a precondition on an invisible object yields "stateMismatch". Clarified that already permits, but does not require, a server to accept changes such as a name exchange whose final state is valid; "atomic" makes this a guarantee. draft-ietf-jmap-conditional-00 Upload with working-group adopted name. draft-gondwana-jmap-conditional-01 Added the "atomic" argument to Foo/set: a method requested with "atomic" is applied as a single unit, with constraints evaluated against the resulting state (enabling changes valid only as a whole, such as an atomic name exchange). Added the "atomicFailure" method-level error (the method's changes could not be committed together) and the "cannotApplyAtomically" method-level error (the server cannot apply the method atomically), both registered with IANA. A server MUST NOT apply an atomic method's changes only in part. Generalised "ifUnchangedBy" to allow a precondition on any object of the method's type, including objects the method does not itself create, update, or destroy (for example a parent or container). Changed the effect of a failed precondition from a per-object "stateMismatch" SetError to a method-level "stateMismatch" error that rejects the whole method and makes no change; a client wanting independent application uses separate method calls. Removed the bespoke per-object SetError definition in favour of Core's method-level "stateMismatch", optionally carrying the "failed" ids. draft-gondwana-jmap-conditional-00 Initial proposal.