The TLSv1.3 protocol eliminated renegotiation. The information in this section is only applicable to TLSv1.2 and earlier protocol versions.

Starting a new handshake negotiation inside of an existing secure session is called renegotiation. There are two properties that determine System TLS renegotiation characteristics.

Multiple reasons exist for an application to use renegotiation. Renegotiation can be started by either the client or server. The application layer might not be aware that a secure session is renegotiated at the request of a peer.
Note: A GSKit System TLS application uses gsk_secure_soc_misc() to initiate renegotiation.

The TLS protocol architecture as defined by their base RFCs contain a flaw with renegotiation. The protocols fail to provide cryptographic verification that a session renegotiation is linked to the existing secure session. Supplemental RFC 5746 defines an optional extension to the base protocols to correct the issue.

Since RFC 5746 is an addition to a previously defined protocol, not all TLS implementations currently support it. Some TLS implementations have not or cannot be updated to support RFC 5746. To allow for business continuity/interoperability during various stages of the transition, two renegotiation properties exist to specify the renegotiation mode.

TLS Renegotiation Mode

The System TLS default requires RFC 5746 semantics are used for all renegotiated handshakes. The default mode can be changed with option renegotiation on the System Service Tools (SST) Advanced Analysis command TLSCONFIG.

The mode can be set to allow all unsecure renegotiations or to allow only abbreviated unsecure renegotiations. Use these modes only after careful consideration. TLSCONFIG option h displays the help panel that describes how to set the TLS renegotiation mode

A mode exists to disable all peer initiated handshake renegotiation. This mode prevents secure (RFC 5746 semantics) and unsecure renegotiation. This mode can result in interoperability issues for applications that require the use of renegotiation. Locally initiated secure renegotiation such as gsk_secure_soc_misc() is still allowed in this mode.

TLS Extended Renegotiation Critical Mode

Extended Renegotiation Critical Mode determines when System TLS requires all peers provide the RFC 5746 renegotiation indication during initial session negotiation.

To completely protect both sides of the secure session against the renegotiation weakness, all initial negotiations must indicate support for RFC 5746. This indication can be in the form of the “renegotiation_info” TLS Extension or the Signaling Cipher Suite Value (SCSV) as defined by RFC 5746.

Critical mode is disabled by default to maintain interoperability with TLS implementations that do not implement RFC 5746. When critical mode is enabled, System TLS is restricted to negotiating with systems that implement RFC 5746. The restriction exists even if neither side supports or uses renegotiation. If it is determined that all System TLS peers support RFC 5746, then this mode should be changed to be enabled.

The default extended renegotiation critical mode can be changed with the System Service Tools (SST) Advanced Analysis command TLSCONFIG. There is a property for client applications, TLSCONFIG option rfc5746NegotiationRequiredClient, and a separate property, TLSCONFIG option rfc5746NegotiationRequiredServer, for server applications.

System TLS always sends the “renegotiation_info” TLS Extension or the SCSV in the ClientHello. SCSV is sent only if no other extensions are part of the ClientHello.