Signature and hash algorithms

The GSK_TLS_SIG_ALG_PAIRS setting indicates the list of signature and hash algorithms that are supported by the client or server consisting of one or more 4-character values in order of preference for use in digital signatures of X.509 certificates and TLS handshake messages. The client sends its supported list in the signature algorithms extension to the server which is used to guide selection of the server's certificate and internal hashing of TLS V1.3 handshake messages. If the server is enabled for client authentication, the server sends its supported list to the client which is used to guide selection of the client's certificate and internal hashing of TLS V1.3 handshake messages.

If the optional GSK_CERT_TLS_SIG_ALG_PAIRS setting is specified, it indicates the list of hash and signature algorithm pair specifications that are supported by the client or server as a string consisting of one or more 4-character values in order of preference for use in digital signatures of X.509 certificates. The client sends is supported list in the certificate signature algorithms extension to the server which is used to guide selection of the server's certificate. If the server is enabled for client authentication, the server sends its supported list to the client which is used to guide selection of the client's certificate. If specified, the GSK_TLS_CERT_SIG_ALG_PAIRS setting overrides the GSK_TLS_SIG_ALG_PAIRS setting when checking the digital signatures of the remote peer's X.509 certificates. Generally, it is not necessary to specify the GSK_CERT_TLS_SIG_ALG_PAIRS setting as GSK_TLS_SIG_ALG_PAIRS setting is used for the supported signature and hash algorithms for certificates and TLS handshake messages.

The hash and signature algorithm that is selected for signing the TLS V1.3 handshake messages are determined in the following manner:
  • A list of the signature algorithms that are in common between the client and server lists is constructed. This common signature algorithm list is in the order of the remote partner's list.
  • Based upon the local certificate that is used during the handshake, an appropriate signature is selected from the common list.
    • An ECC certificate uses an ECDSA signature algorithm.
    • An RSA certificate signed with an RSA or an RSASSA-PSS signature algorithm uses an RSASSA-PSS signature algorithm.
  • If an appropriate signature algorithm is not found in the common list on the server side, the server will select one from its list. See Table 1 for the preferred signature algorithms that is used.
  • If the client sends a certificate to the server and an appropriate signature algorithm is not found in the common list, the TLS V1.3 handshake fails.
Table 1. TLS V1.3 preferred signature algorithm to use for signing handshake messages based upon certificate types
Certificate type Preferred Signature Algorithm
RSA certificate with a signature algorithm of RSA with SHA-1 (Key sizes 2048 and larger) 0804 - SHA-256 with RSASSA-PSS
RSA certificate with a signature algorithm of RSA with SHA-256 (Key sizes 2048 and larger) 0804 - SHA-256 with RSASSA-PSS
RSA certificate with a signature algorithm of RSASSA-PSS with SHA-256 (Key sizes 2048 and larger) 0804 - SHA-256 with RSASSA-PSS
RSA certificate with a signature algorithm of RSA with SHA-384 (Key sizes 2048 and larger) 0805 - SHA-384 with RSASSA-PSS
RSA certificate with a signature algorithm of RSASSA-PSS with SHA-384 (Key sizes 2048 and larger) 0805 - SHA-384 with RSASSA-PSS
Start of changeRSA certificate with a signature algorithm of RSA with SHA-512 (Key sizes 2048 and larger)End of change 0806 - SHA-512 with RSASSA-PSS
Start of changeRSA certificate with a signature algorithm of RSASSA-PSS with SHA-512 (Key sizes 2048 and larger)End of change 0806 - SHA-512 with RSASSA-PSS
Any ECC secp256r1 certificate 0403 - SHA-256 with ECDSA
Any ECC secp384r1 certificate 0503 - SHA-384 with ECDSA
Any ECC secp521r1 certificate 0603 - SHA-512 with ECDSA
Note: Any other certificate types that are not listed in Table 1 are not supported with TLS V1.3. ECC certificates that have a key size smaller than 256 bits and RSA certificates that have a key size smaller than 2048 are not supported with TLS V1.3.

On the server side, certificates that use SHA-1 signature algorithms for RSA and ECDSA will be ignored unless the client has included SHA-1 with RSA and SHA-1 with ECDSA in its signature algorithm list or certificate signature algorithm list. The server uses these algorithms as a last resort.

If the application is only enabled for TLS V1.3, hash and signature algorithms that are not allowed in TLS V1.3 (for example, 0202) are not sent to the partner and are quietly ignored.