Institute of Information Engineering, Chinese Academy of Sciences

liuhongbo@iie.ac.cn

Institute of Information Engineering, Chinese Academy of Sciences

zhangqionglu@iie.ac.cn

Institute of Information Engineering, Chinese Academy of Sciences

lizhaoxuan@iie.ac.cn

North China Electric Power University

120262127028@ncepu.edu.cn

Institute of Information Engineering, Chinese Academy of Sciences

luxianhui@iie.ac.cn

SSH KEM user authentication This document specifies publickey-kem, a Key Encapsulation Mechanism (KEM)-based user authentication method for the Secure Shell (SSH) protocol. The server encapsulates a fresh shared secret to an authorized client public key. The client demonstrates possession of the corresponding private key by returning a session-bound confirmation value. The method does not modify SSH transport or server host authentication. Initial authentication algorithm profiles using ML-KEM and HMAC-SHA256 are also specified.

Introduction SSH separates transport-layer security from user authentication and the connection protocol . The widely used publickey user authentication method proves possession of an authorized private key through a digital signature. Post-quantum signature algorithms can be accommodated by that method , while a KEM provides a different means of proving possession. A server encapsulates a secret to an authorized public key, and the client demonstrates knowledge of the corresponding secret through decapsulation and key confirmation. This document defines publickey-kem as a separate SSH user authentication method. It preserves SSH's public-key credential and authorization model while replacing the client signature with a challenge-response confirmation bound to the SSH session and authentication request. The method introduces no transport-layer changes and can coexist with other SSH authentication methods. It builds on the KEMUAuth construction described in . KEM-based authentication has also been studied for TLS and EDHOC . The method defined here is independent of the choice of KEM and confirmation function. Section 5 specifies initial authentication algorithm profiles using ML-KEM and HMAC-SHA256. Additional profiles can be defined within the same authentication method. This specification concerns SSH client user authentication. SSH transport-layer migration mechanisms, including and , are defined separately.

Conventions and Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. SSH data types, including byte, string, and name-list, follow . SSH_string(x) denotes the SSH string encoding of the octet sequence x, consisting of a four-octet length in network byte order followed by x. A string carrying a public key blob includes its own length prefix, independently of the SSH-encoded fields inside the blob. A Key Encapsulation Mechanism (KEM) provides three algorithms. (pk_C, sk_C) = KeyGen() (K_C, ct_C) = Encap(pk_C) K_C' = Decap(sk_C, ct_C) KeyGen produces a public encapsulation key and a corresponding private decapsulation key. Encap produces a shared secret and a ciphertext using the public key. Decap recovers the shared secret using the private key and ciphertext. For honestly generated keys and encapsulations, correctness requires K_C' = K_C, except with any permitted correctness error specified by the KEM. Profile-specific input validation and decapsulation behavior are defined by the selected algorithm profile. The following notation is used throughout this document.

• pk_C and sk_C denote the client's long-term KEM encapsulation and decapsulation keys.
• ct_C denotes the KEM ciphertext produced for a single authentication attempt.
• K_C denotes the shared secret produced by the server's encapsulation. The client recovers the corresponding secret by decapsulation; in the normal exchange both parties use the same K_C.
• sid denotes the SSH session identifier established by the first transport-layer key exchange .
• alg denotes the complete authentication algorithm identifier, which selects the KEM, confirmation function, and profile parameters.
• public_key_blob denotes the SSH-encoded public key object, beginning with its public key format identifier.
• Confirm_alg denotes the confirmation function selected by alg.
• label denotes the domain-separation octets defined in Section 4.2. ctx denotes the encoded authentication context, and MAC_input denotes the complete input to Confirm_alg.
• ca denotes the client authentication value.

An authentication attempt is one execution of publickey-kem, from its initial request to completion or abandonment. An authorized key is one permitted by server policy for the requested user and service. A valid KEM public key is not necessarily authorized.

KEM-based User Authentication

Method Overview The client holds a long-term KEM private key, and the server associates the corresponding public key with an SSH user. After receiving a publickey-kem request, the server validates the selected algorithm and public key, checks authorization, and encapsulates a fresh shared secret. The client checks the challenge, decapsulates the ciphertext, and returns a confirmation value bound to the SSH session and the authentication attempt. The server verifies the response and applies the normal SSH authentication result rules.

KEM-based SSH user authentication exchange Client Server | | | SSH_MSG_USERAUTH_REQUEST (50) | | user, service, "publickey-kem", alg, public_key_blob | |------------------------------------------------------------> | Validate algorithm, key and authorization | | (K_C, ct_C) = Encap(pk_C) | | | | SSH_MSG_USERAUTH_KEM_CHALLENGE (60) | | alg, public_key_blob, ct_C | <------------------------------------------------------------| | Check echoes and ciphertext | | K_C = Decap(sk_C, ct_C) | | ca = Confirm_alg(K_C, MAC_input) | | | | SSH_MSG_USERAUTH_KEM_RESPONSE (61) | | ca | |------------------------------------------------------------> | Verify ca and authorization policy | | | | SSH_MSG_USERAUTH_SUCCESS / FAILURE | <------------------------------------------------------------|

Failure may occur before or after the challenge. All messages are carried inside the protected SSH transport channel. The KEM shared secret is used for client authentication and is not incorporated into SSH transport key derivation. The SSH transport key exchange and KEM user authentication algorithms are selected independently. Since the server also knows the shared secret, the confirmation value is not a transferable client signature.

Authentication Algorithm Profiles An authentication algorithm profile specifies a KEM, a public key format, the ciphertext representation, a confirmation function and output length, and any required cryptographic validation. The alg field identifies the complete profile. KEM and confirmation functions are not negotiated separately. The public key format identifier is distinct from the authentication algorithm identifier. Different authentication algorithms MAY share a public key format when their profiles explicitly permit it. Each profile MUST define the information required for independent implementations to encode, process, and verify an authentication attempt.

Protocol Specification

Messages and Encodings The publickey-kem method uses the SSH user authentication framework of and defines two method-specific messages. SSH_MSG_USERAUTH_KEM_CHALLENGE (60) is sent by the server, and SSH_MSG_USERAUTH_KEM_RESPONSE (61) is sent by the client. The numbers are interpreted within the active authentication method. Section 6 of describes method-specific messages as server-to-client, while defines the client-to-server SSH_MSG_USERAUTH_INFO_RESPONSE using number 61. This method follows that established precedent for a client response. The client begins an authentication attempt with the following request. byte SSH_MSG_USERAUTH_REQUEST /* 50 */ string user name string service name string "publickey-kem" string authentication algorithm string public key blob The authentication algorithm identifies the complete profile. The public key blob is an SSH-encoded object containing the public key format identifier followed by the profile-specific key fields. The request contains no publickey signature-probe boolean. The server checks that it supports and permits the algorithm, that the public key encoding is valid for the selected profile, and that the credential is authorized for the requested user and service. If a check fails, the server rejects the request using SSH_MSG_USERAUTH_FAILURE. Otherwise, it performs (K_C, ct_C) = Encap(pk_C) and sends the following challenge. byte SSH_MSG_USERAUTH_KEM_CHALLENGE /* 60 */ string authentication algorithm string public key blob string KEM ciphertext The algorithm and public key blob in the challenge MUST be byte-for-byte identical to those in the request. The ciphertext is encoded as an SSH string containing the ciphertext representation specified by the selected profile. Each challenge MUST use a fresh encapsulation. The server MUST NOT reuse a challenge and its shared secret for a different authentication attempt. The client MUST verify that the challenge corresponds to its pending request and that both echoed fields match exactly. It MUST apply the ciphertext input checks required by the profile before decapsulation. If these checks succeed, it decapsulates using the associated private key and sends the following response. byte SSH_MSG_USERAUTH_KEM_RESPONSE /* 61 */ string authentication value The authentication value is ca as defined in Section 4.2. The response does not repeat the user, service, or key fields. They are taken from the pending request and challenge. All on-wire algorithm and public key format identifiers are case-sensitive. The publickey-kem request, challenge, and response formats above contain no extension fields. Implementations MUST reject these messages if they contain unexpected trailing octets. Error handling follows Section 4.3. The server completes the attempt using the existing SSH user authentication result messages . byte SSH_MSG_USERAUTH_FAILURE /* 51 */ name-list authentications that can continue boolean partial success or byte SSH_MSG_USERAUTH_SUCCESS /* 52 */ The interpretation of these results, including partial success, is defined by . This method defines no additional result message.

Confirmation Input and Verification After completing the selected KEM decapsulation procedure, the client computes the authentication value as follows. ca = Confirm_alg(K_C, MAC_input) MAC_input = label || SSH_string(sid) || ctx The label is the raw US-ASCII octet sequence ssh-publickey-kem-client-auth, without a trailing NUL or an SSH string length prefix. The sid is encoded as one SSH string. It is the session identifier established by the first SSH key exchange and does not change following transport rekeying . The context ctx is the concatenation of the following fields in exactly this order, using SSH binary encoding . byte SSH_MSG_USERAUTH_REQUEST /* 50 */ string user name string service name string "publickey-kem" string authentication algorithm string public key blob byte SSH_MSG_USERAUTH_KEM_CHALLENGE /* 60 */ string authentication algorithm string public key blob string KEM ciphertext byte SSH_MSG_USERAUTH_KEM_RESPONSE /* 61 */ The request and challenge portions of ctx contain exactly the transmitted SSH field values, including the outer string encoding of the public key blob and its inner encoded fields. The final byte is the response message type 61. The response's authentication value is excluded. SSH packet lengths, transport padding, packet sequence numbers, and transport authentication tags are also excluded. No additional normalization, separators, or encoding are applied. The server computes the expected authentication value using the shared secret generated by its encapsulation and the request and challenge fields associated with the pending attempt. The server MUST verify that the response belongs to that attempt, has the profile's required length, and equals the expected value. Fixed-length secret confirmation values MUST be compared in constant time. The credential MUST still be authorized when the server accepts the authentication. For the ML-KEM profiles of Section 5, Confirm_alg is HMAC-SHA256. The ML-KEM shared secret is used directly as the HMAC key, and the entire 32-octet output is ca.

State, Failure, and Completion The server MUST bind each pending challenge to the user, service, authentication algorithm, exact public key blob, ciphertext, and the shared secret or expected confirmation value needed for verification. The server MUST maintain no more than one outstanding challenge for the active publickey-kem attempt. A KEM response received without a corresponding pending challenge MUST NOT be accepted as authentication. The following rules apply to an attempt. Authentication Attempt State and Failure Handling

Condition	Handling
Unsupported algorithm, invalid key encoding, or unauthorized credential	Reject the initial request without encapsulation.
Challenge echoes do not match, or ciphertext fails required client-side input checks	The client MUST NOT send a KEM response for that challenge.
Authentication value has the wrong length or fails verification	Reject the attempt using SSH_MSG_USERAUTH_FAILURE with partial success set to FALSE.
A new SSH_MSG_USERAUTH_REQUEST interrupts the pending challenge	Abandon the pending attempt and process the new request under . Do not send a failure response solely for the aborted attempt.
A KEM response arrives without an active publickey-kem challenge	Treat as a protocol-state error. The server SHOULD disconnect and MUST NOT authenticate the client.

The protocol-state error rules above apply while SSH user authentication is in progress. After sending SSH_MSG_USERAUTH_SUCCESS, the server SHOULD silently ignore subsequent authentication-related messages, as specified in Section 5.3 of . Clients and servers MUST follow the request sequencing and interruption rules of . In particular, a client can abandon an interactive authentication exchange by starting another request as permitted by that specification. The server MUST discard the old challenge when that happens. When the user name or service name changes, the server MUST also clear accumulated authentication state, or disconnect if it cannot, as required by . A challenge is consumed when its response has been processed, regardless of whether verification succeeds. A consumed challenge MUST NOT be accepted again. Pending challenge state MUST be discarded following completion, abandonment, authentication timeout, or connection termination. Secret intermediate values SHOULD be erased when no longer needed. If publickey-kem succeeds and no other factor is required, the server sends SSH_MSG_USERAUTH_SUCCESS. If this method succeeds but additional authentication is required, the server sends SSH_MSG_USERAUTH_FAILURE with partial success set to TRUE. A failed publickey-kem attempt uses partial success FALSE. These results follow . Malformed method-specific payloads and recognized messages received in an invalid authentication state MUST NOT result in successful authentication. A malformed payload may be treated as a protocol error. Handling of truly unrecognized SSH message numbers follows the applicable SSH_MSG_UNIMPLEMENTED rules of . For profiles with implicit rejection, decapsulation MUST follow the selected KEM specification. No internal decapsulation consistency outcome is sent as a separate SSH error. Subsequent confirmation failures use the normal authentication failure rules. The method introduces no KEM-specific failure message.

ML-KEM Authentication Profiles

Algorithms and Parameters This section instantiates publickey-kem using ML-KEM and HMAC-SHA256 . The following authentication algorithms and public key formats are defined. ML-KEM Authentication Algorithm Profiles

KEM	Authentication algorithm	Public key format	Public key (octets)	Ciphertext (octets)
ML-KEM-512	ssh-mlkem-512-hmac-sha256	ssh-mlkem-512	800	768
ML-KEM-768	ssh-mlkem-768-hmac-sha256	ssh-mlkem-768	1184	1088
ML-KEM-1024	ssh-mlkem-1024-hmac-sha256	ssh-mlkem-1024	1568	1568

The table gives raw FIPS 203 encapsulation-key and ciphertext sizes, excluding SSH field-length prefixes. Every profile uses a 32-octet ML-KEM shared secret and a 32-octet confirmation value. Implementations of publickey-kem SHOULD support ML-KEM-768 and MAY also support ML-KEM-512 or ML-KEM-1024. These requirements do not define automatic algorithm selection or preference ordering. Each authentication algorithm identifies its ML-KEM parameter set, public key format, and HMAC-SHA256 confirmation function. An implementation MUST NOT substitute a different KEM parameter set or confirmation function under the same authentication algorithm name. The public key format MUST match the selected algorithm.

Public Keys and Ciphertexts The public key blob field of Section 4.1 contains an SSH-encoded object with exactly the following internal fields. string public key format identifier string ML-KEM encapsulation key The first field contains the public key format identifier from Section 5.1. The second contains the raw ML-KEM encapsulation key specified by . Extra fields and trailing octets are not permitted. The complete two-field object is carried as the contents of the outer string public key blob field in the authentication request and challenge. The server MUST ensure that an ML-KEM encapsulation key satisfies the input checks of Section 7.2 of before encapsulation. Key validation MAY be performed when the credential is provisioned, provided the result remains associated with the same unmodified key. Authorization for the user and service is a separate requirement. The challenge ciphertext is the raw output of ML-KEM.Encaps, with the length specified in Section 5.1. The client MUST ensure that its decapsulation key has passed the required input checks of Section 7.3 of . A previously validated key need not be revalidated when its integrity is maintained. The client MUST check the ciphertext length on every decapsulation attempt and MUST reject a ciphertext with an incorrect length. For a correctly sized ciphertext and a validated decapsulation key, the client runs ML-KEM.Decaps according to , including implicit rejection. No separate implicit-rejection indicator is transmitted in SSH.

Confirmation Function For each profile in this section, the confirmation function is defined as follows. K_C = 32-octet ML-KEM shared secret ca = HMAC-SHA256(K_C, MAC_input) MAC_input is specified in Section 4.2. The ML-KEM shared secret is used directly as the HMAC-SHA256 key without an additional hash or key derivation step, consistent with key-confirmation guidance in . The complete 32-octet output is encoded as the contents of the string authentication value field. The shared secret is used solely for this authentication confirmation. Neither the secret nor the authentication value enters SSH transport key derivation. Additional authentication algorithm profiles MAY specify other KEMs or confirmation functions. Each new profile MUST specify key and ciphertext encodings, validation rules, the exact confirmation computation, and the confirmation output length. The protocol messages and state rules of Section 4 remain unchanged unless another specification explicitly extends them.

Deployment and Interoperability SSH clients and servers can deploy publickey-kem alongside existing authentication methods. Servers associate authorized KEM public keys with users according to local policy. The public key encoding is defined in Section 5, while credential provisioning and storage are implementation-specific. SSH certificate and agent integrations are outside the scope of this specification. Servers can advertise publickey-kem in the authentications that can continue list of . This list contains authentication method names, not the KEM authentication algorithms of Section 5. A client selects an algorithm for which it holds a corresponding private key, and the server checks support and authorization upon receiving the request. The server-sig-algs extension of describes algorithms accepted for the publickey authentication method and does not negotiate publickey-kem profiles. The publickey-kem method can serve as one factor in multi-method authentication. Servers can require it together with publickey, password, or other methods using the policy and partial-success mechanisms of . A server that does not support publickey-kem rejects that method through the normal SSH authentication mechanism. Clients and servers configured to require KEM-based authentication MUST NOT bypass that requirement by falling back to a disallowed method. Section 7.3 discusses the associated downgrade considerations.

Security Considerations

Authentication and Cryptographic Assumptions The publickey-kem method relies on an authorized long-term KEM public key, unambiguous confirmation input encoding, and an authenticated, integrity-protected SSH transport channel. The initial profiles use ML-KEM and HMAC-SHA256 . Under the cryptographic assumptions established in , the KEMUAuth construction achieves mutual authentication security in the ACCE model against classical adversaries. The result also holds against quantum adversaries when the required assumptions are satisfied in the post-quantum setting. These assumptions include an IND-CCA-secure client KEM, a secure authentication PRF, and the required security of SSH transport key exchange, server host authentication, key derivation, and channel protection. The proof builds on the SSH ACCE analysis in and addresses protocol-level security rather than implementation-level risks such as side-channel attacks. The confirmation input binds the proof to the SSH session identifier and the exact user, service, algorithm, public key, and KEM challenge of the authentication attempt. Fresh encapsulation and one-time challenge handling prevent a recorded response from being reused as a valid response to a different challenge. The server MUST verify both the confirmation value and the credential's authorization before accepting the method. Exposure of the client's long-term decapsulation key can enable impersonation while the matching public key remains authorized. The confirmation secret is not used as a transport key. Confidentiality, forward secrecy, and post-quantum protection of the SSH channel depend on the negotiated transport and host-authentication mechanisms rather than on publickey-kem alone.

Privacy, Deniability, and Resource Use The server also knows the encapsulated shared secret and can compute the same confirmation value as the client. The value is therefore not a publicly verifiable client signature and does not provide transferable proof of client origin. A server's decision to issue a challenge can reveal whether a supplied public key is authorized for a user. The existing publickey method permits a similar key-acceptability query . This method does not conceal key authorization status. Servers SHOULD bound authentication attempts, encapsulation work, and pending-state lifetime, consistent with the limits and timeout guidance in . Failure diagnostics should avoid disclosing unnecessary account and credential information. Implementations of the ML-KEM profiles MUST preserve the input-validation and implicit-rejection behavior of . Constant-time confirmation comparison is required by Section 4.2. These cryptographic handling rules do not themselves prevent account or credential enumeration caused by distinguishable authorization responses.

Migration and Downgrade The publickey-kem method can coexist with existing SSH authentication methods. The security of the resulting deployment depends on the authentication methods and combinations the server accepts. A policy accepting either KEM-based authentication or a different method permits the latter even when KEM authentication is available. User authentication messages are sent inside the established SSH transport channel. The algorithm identifier is checked against the challenge and included in the confirmation input. These bindings prevent an attacker who cannot compromise the protected channel from silently substituting a different algorithm during a valid publickey-kem attempt. They do not by themselves select the strongest available algorithm or prohibit a separate authentication attempt using another method. Clients and servers configured to require publickey-kem or a particular authentication algorithm MUST NOT weaken that requirement merely because it is unavailable. Servers MUST enforce all locally required authentication factors. The security of the transport key exchange and host authentication remains governed by the algorithms and policies for those layers.

IANA Considerations This document requests that IANA register publickey-kem in the SSH "Authentication Method Names" registry, with this document as its reference. The following entries are proposed for the SSH "Public Key Algorithm Names" registry. Proposed Public Key Algorithm Registrations

Public Key Algorithm Name	Public Key Format	Reference
ssh-mlkem-512-hmac-sha256	ssh-mlkem-512	This document
ssh-mlkem-768-hmac-sha256	ssh-mlkem-768	This document
ssh-mlkem-1024-hmac-sha256	ssh-mlkem-1024	This document

These names designate the KEM user-authentication profiles of Section 5, rather than SSH transport key-exchange methods or signature algorithms. The registration policies are specified by as updated by . The challenge and response use message numbers 60 and 61 within the existing user-authentication method-specific message space . No new global SSH message number allocation is requested. HMAC-SHA256 is part of the authentication profiles defined here and is not an SSH transport MAC algorithm. No entry in the "MAC Algorithm Names" registry is requested.

Normative References National Institute of Standards and Technology National Institute of Standards and Technology Informative References National Institute of Standards and Technology ACM CCS 2020 IEEE Symposium on Security and Privacy, 2025 IEEE ICNP 2026 (extended manuscript available on arXiv)

ML-KEM-768 Interoperability Test Vector This appendix provides one deterministic successful authentication example for the ssh-mlkem-768-hmac-sha256 profile. Hexadecimal strings represent raw octets. These fixed inputs are for testing only and are not suitable for operational key generation or encapsulation. The user name is alice, the service name is ssh-connection, the authentication method is publickey-kem, and the public key format is ssh-mlkem-768. The values d, z, and m are deterministic ML-KEM inputs defined in . The 32-octet sid is a synthetic test value rather than the result of a live SSH key exchange. d=000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f z=202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f m=404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f sid=606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f The deterministic ML-KEM-768 operations and their relevant outputs are: (ek, dk) = ML-KEM.KeyGen_internal(d, z) (K_C, ct) = ML-KEM.Encaps_internal(ek, m) K_C=9cddd089ffe70e3996e76f7c8d06746df34d07e8657bc0fcf2bb0e1c3084aea1 ca=c5144f336c20cc54210c3156da6b19d444c36cfdbc1250a485ef04916e743505 The request and challenge are the complete SSH message payloads defined in Section 4.1, beginning with the message-type bytes 50 and 60, respectively, and excluding SSH transport packet framing. The 0x3d octet denotes message type 61. The label is raw ASCII with no length prefix or terminating NUL. public_key_blob = SSH_string("ssh-mlkem-768") || SSH_string(ek) ctx = request || challenge || 0x3d MAC_input = ASCII("ssh-publickey-kem-client-auth") || SSH_string(sid) || ctx ca = HMAC-SHA256(K_C, MAC_input) response = 0x3d || SSH_string(ca) The following sizes and SHA-256 digests allow independent implementations to verify the intermediate encodings. Digests are computed over raw binary octets, not hexadecimal text. ek (1184 octets), SHA-256: 0b7934c83125c788995e2ba6bd761e33046b3e40571be53e023309a29f398cc9 ct (1088 octets), SHA-256: dbf4e9aa48b078ad46ec1c9c47bda8c2d2fec9d0e7a21bd48d2238a2abedb856 public_key_blob (1205 octets), SHA-256: 1c8bab6cc89a9c0564dfa02b7c6368580443a14e5d25d9cec1dd755a5646d589 request (1283 octets), SHA-256: 930a740e341f8a3a73933942047bbc5028a03e9ee8c61a5ce2820a22c6f64c37 challenge (2331 octets), SHA-256: bf7faf5ef0039b13d747828c5519f7e7806e135949cd5ac3febb07b8449969e0 ctx (3615 octets), SHA-256: aae32626c00541345d1fd7863f81d33b7b4f83a983cb4bd800eb005670047d05 MAC_input (3680 octets), SHA-256: 3dc89cd08296d099c699d0e5393519032cc609c2ca5e195a8c05f84372dfe1b4 response (37 octets), SHA-256: eedacf149c61bd836cba35d045de603240ec22ca8e65fb5ecb6d4a6d427148c5

Acknowledgements This work was supported in part by the National Cryptologic Science Fund of China under Grant 2025NCSF01008.