Understanding Certificate Cross-Signing

Foundationally, SSL/TLS provides two things: encryption (by way of symmetric and asymmetric cryptography) and validation of trust. Certificate Authorities are entities whose primary role is to responsibly issue digital certificates after performing validation that the request is coming from somebody authorised to make such a request for a given domain name. Clients (which can be web browsers, instances of java, or arbitrary pieces of software making requests) are shipped with some number of trusted root certificates. Root certificates by definition are self-signed, and a client is already aware of them. When it then makes a request to an endpoint secured via SSL/TLS, the client attempts to construct a valid “chain” (through iteratively comparing digital signatures) from a root certificate it is already aware of, to the public key inside of the node certificate presented by the endpoint it is connecting to. If the client is able to calculate a successful path of trust from one of the root CAs it’s aware of, through one or more intermediate certificates, and down to the node certificate, it makes the decision to trust the endpoint. If it cannot perform this feat, it throws an error and aborts the connection attempt.

How then, astute readers might ask, are clients made aware of intermediate certificates? Practically speaking, in addition to root certificates, most clients also ship with some number of intermediate certificates of which they are aware. Additionally, when configuring an endpoint to serve a public key, administrators can optionally include an intermediate certificate as well. Because the client still has to construct its own path from a trusted root down to the node certificate, having the intermediate provided by the endpoint under interrogation is perfectly fine. It’s worth noting that an administrator can ALSO serve a root certificate (because the system has no way to know that it isn’t also an intermediate for some other valid path), but this configuration is incorrect. This is harmless, except for a couple of kb of unnecessary information being sent with every request.

What then, is cross-signing? Cross-signing is simply when multiple valid paths exist between a root certificate and a node certificate. This can be advantageous for a number of reasons. Sometimes, a certificate in the chain expires. While it would be nice if every piece of software was frequently updated and shipped an up-to-date copy of the root CA store, this is not the case in the real world. Many devices, for a number of reasons, are not subject to regular updates. Strategically cross-signing intermediate certificates from an older (therefore more likely to be present on the larger subset of devices in the wild) root certificate, “buys some time”. Hopefully by the time the older-still root certificate expires, the device will have been replaced. We can dream, right? Cross-signature is also useful when CAs need to perform maintenance. When vulnerabilities in the SHA-1 hashing algorithm were found, CAs issued new intermediate certificates which were able to cross-sign for yet more intermediates lower down in the chain. This allows, at least in theory, trust to continue seamlessly. In reality, this switcharoo working in practice is highly dependent on both software implementation and administrator configuration.

Software implementations for constructing a chain of trust vary widely. Especially for older clients, it is not uncommon for naïve implementations to refuse to connect after it constructs a once-valid path. Note that this situation is distinctly different from constructing a valid path or being unable to construct a valid path. If there is an expired intermediate in the chain, the system realises that this path is a valid path, and THEN checks to make sure all certificates in the mix are still in good standing, this can cause problems. Specifically, at this point a clever implementation would continue to search for OTHER valid paths instead of giving up. This is shockingly still a very common problem today, as constructing a test for this relatively uncommon situation is a lot of work! People often rely on libraries to handle these low-level tasks, and these libraries may be rarely updated, even for software that ships regular updates for other key areas. This is an example of technical debt, and is very difficult to plan for.

Another common situation one might fall into is that an endpoint you are responsible for correctly serves intermediate certificates to help inform systems which might not be aware of those particular intermediates, but are aware of the root certificate. Then one day, the intermediate certificate expires. Perhaps the client implementation is such that it is clever enough to handle the situation above, but yet still gives weight to what the endpoint it’s connecting to is serving beyond what is warranted. (In the case that the system is aware of a different valid path, but because an invalid path is being “suggested” by the endpoint, it stops trying to find a valid chain of trust). To conflate matters even further, IIS on Windows is a bit cheeky. Unlike most server software, it does NOT allow you to directly specific which intermediate certificate is served, but rather chooses one based on somewhat obtuse logic, often serving an expired intermediate.

We can use the handy OpenSSL Command to check for what certificates are being served by an endpoint:

Part of the response returned will include base64 encoded certificates in the chain. These can be decoded in order to determine their expiration and thumbprint.

Unfortunately, there is no great way to monitor for this situation. That would presuppose being aware of every possible client state, which is a huge undertaking even for a small network with only a few different types of software. (Good luck asking your vendor this extremely specific question, you might have better luck with a decompiler!) Even though there are server-side “hints” which drive client behaviour, this is inherently a client-side problem and in the purview of the software developer to fix. What then are some ways to attempt to stave off problems? For systems whose uptime is critical, it is a wise choose to choose to purchase certificates from a signer whose PKI is as “flat” as possible. Choosing from some of the bigger names (even though they are more expensive) are older, and more likely to be trusted by bespoke client implementations. Additionally, when installing certificates, checking to make sure that the intermediates do not expire before the node certificates is a good practice.