Eight Bits of Trust: eMudhra's CA keys cracked in milliseconds

Bugzilla bug in question: https://.mozilla.org/showbug.cgi?id=2083449

• LoganDark · 20 hours ago

  https://bugzilla.mozilla.org/show_bug.cgi?id=2083449

Test your browser and update your system's certificates: https://emudhra.we-have-a-problem.com/

Never attribute to malice what can be explained by ignorance? In this case I wonder if this was an intentional wide open doorway to fraud

• MiroslavPokorny · 19 hours ago

  This wasnt malice its was laziness. They literally were too lazy to type more than just a single '4'.

  • unsupp0rted · 17 hours ago

    Either that or they intentionally wanted to create a publicly-accessible backdoor, with plausible deniability of laziness

I thought CAs automatically filtered out really low keys. There's a great Suckerpinch video ( https://youtu.be/M1si1y5lvkk ) about trying to purposely make the least secure HTTPS page possible, and Tom initially tried to choose a laughably-low number as the key, but the CA automatically rejected it.

That's some criminal negligence right there. 8 bits is crazytalk!

• joshuat · 13 hours ago

  “4” hey chatgbg what is a “private key”

  • Hamuko · 12 hours ago

    Maybe they were counting on the fact that usually the right answer to “I’m thinking of a number between 1 and 10” is “7”.

  • Agentlien · 12 hours ago

    As someone from Gothenburg, Sweden (often referred to as gbg) I shall answer: A private key is a tiny number you don't need to share publicly because it is easy enough to guess.

• bombcar · 11 hours ago

  Isn't 4 simply 2 bits? Silly 2-bit outfit. Maybe it's 3 bits.

  • patrickthebold · 11 hours ago

    I think the way that the bits are counted is based on the pool from which the number was randomly drawn. So it's possible they had 256 bits of entropy but got really unlucky.

    • thesz · 8 hours ago

      Their other keys are also in 0..255 range. This is definitely not "really unlucky." I guess they used (x `mod` 256) instead of (x `mod` (2^256)).

Of COURSE it's easy to crack, 4 isn't even prime!

• joshuat · 13 hours ago

  It’s really impressive no one caught their three previous private key values

  • bcvvvy · 13 hours ago

    I suspect they did but it wasn't publicised.

• DavidSJ · 12 hours ago

  In their defense, they were probably targeting the Mersenne prime 2^2 - 1 and just made a typo.

  • throw0101a · 12 hours ago

    > In their defense, they were probably targeting the Mersenne prime 2^2 - 1 and just made a typo. There are two hard problems in computer science: naming things, cache invalidation, and off-by-one errors.

    • zapkyeskrill · 10 hours ago

      Two, and only two?

      • amoss · 10 hours ago

        You have almost understood the joke.

      • floydnoel · 4 hours ago

        two is off-by-one from three btw

      • throw0101a · 1 hour ago

        Well: There are actually only two hard problems in computer science: 0) Cache invalidation 1) Naming things 4) Asynchronous callbacks 2) Off-by-one errors 3) Scope creep 6) Bounds checking * https://news.ycombinator.com/item?id=38806257 Also: https://martinfowler.com/bliki/TwoHardThings.html

• delibes · 12 hours ago

  https://xkcd.com/221/

  • lta · 11 hours ago

    It's been a while since this one popped up. It's surprising how many of those doodles are surviving the test if times, in the midst unexpected ways

• walrus01 · 12 hours ago

  Hey, that's the same combination I have on my luggage!

  • wowczarek · 10 hours ago

    Good job the Earth doesn't have an air shield protected by this byte or we'd be Skroob'd.

• wolfi1 · 11 hours ago

  ok, it's joke. but for those who don't get it: elliptic curves don't need primes

• chompychop · 11 hours ago

  They should have gone with 57. Everyone knows it's prime.

  • kachnuv_ocasek · 11 hours ago

    Alex? You're alive after all?

  • bee_rider · 8 hours ago

    I wonder if anyone wants to over-analyze this joke to come up with the best candidates for the number? 57 falls too easily to the quick divisibility by 3 trick, IMO. (I realize over-analyzing jokes always makes them unfunny but we can pick the best number here and nobody has to know).

    • samatman · 6 hours ago

      Me! I'm in! Most-nearly-prime number under 100 is.... 91. 13 x 7. You've either memorized that or you're going to have fun deciding it's not prime. And if you did times tables up to 13, congratulations, you're special.

      • sejje · 3 hours ago

        7 is an easy check, not because i did them up to 13. because 10x7 = 70, i'm left with 21. everyone knows 21 is divisible by 7.

      • bee_rider · 2 hours ago

        I think 91 is a strong contender. Sejje has a way of working it out, but of course any small non-prime will have some trick to it.

• Zeetah · 9 hours ago

  The number 4 is very famous. https://xkcd.com/221/

• ohuf · 8 hours ago

  it doesn't have to be and it actually never is: this isn't RSA, it's an elliptic curve algorithm. if you'd have read the article, you'd have understood this

Related: https://news.ycombinator.com/item?id=50166579 “eMudhra intermediate CA compromise” | 947 points | 19 hours ago | 408 comments

It feels like they chose the private keys by hand.

• PalmPilotProMax · 12 hours ago

  Artisanal cryptography. Hand-picked private keys salted with organically sourced nonces.

  • donandon · 12 hours ago

    Made with love

  • tclancy · 12 hours ago

    Brooklyn-based. Guy in overalls and a waxed mustache hands out the bits.

    • bigfishrunning · 9 hours ago

      Don't forget the leather apron, it's a messy job

  • account42 · 11 hours ago

    Sharing is caring after all.

• muvlon · 12 hours ago

  My bet is on one of - A sequence number being accidentally used as the private key or - The private key being generated securely but then accidentally truncated to 8 bits (e.g. writing a single char into a zeroed buffer when you should've memcpy'd from a char*). The sequence number idea is supported by the fact that two of the G3B private keys are consecutive numbers. I ran a quick Monte Carlo simulation and if I got it right, sampling 8 keys from [0,255] uniformly at random (as in the truncation scenario), you'd expect them to contain 2 consecutive numbers only about 18% of the time.

  • account42 · 11 hours ago

    All the numbers also have digits going from left to right on a keyboard. I think the manual "random" input theory is plausible.

    • muvlon · 11 hours ago

      Oh wow, that is even more damning. Only 146 out of the possible 256 keys have this property. You only have about a 1% chance to get that 8 times in a row (A little more than 1% if sampling with replacement and a little less if sampling without. You'd expect them to at least not use the exact same key twice but at this point???).

  • Avamander · 11 hours ago

    Some error code being used? FIPS mode gone wrong?

• walrus01 · 12 hours ago

  A long time ago one time pads used to be chosen by hand by women picking numbered lottery balls out of a spherical tumbling machine on a table.

• Fabricio20 · 12 hours ago

  The fact that there are more than the "4" one, and all the low numbers are below 384 (384 being the bits for private key) makes me think whoever was creating these thought "Oh a random number between 1-384" let me choose... 4! (156, 230, 115, 56, 66, 237, 155)... https://xkcd.com/221/

• gquere · 10 hours ago

  There's a beginning of an explanation in the mozilla bugrepor t https://bugzilla.mozilla.org/show_bug.cgi?id=2083449

  • mminer237 · 9 hours ago

    Thanks. The relevant quote: > We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)

    • benmmurphy · 8 hours ago

      `a flaw in the code resulted in the second pair of keys being created onboard which were weaker` that sounds like the HSM is capable of generating keys in a range that is smaller than [1..group_order). such a feature makes no sense at all. or is this some kind of weird thing where people want to generate weak keys for 'testing' and HSM vendors have added such a strange feature. its actually satisfying to see someone get burned by this because such a feature should never exist and this is the reason why. if the HSM can generate keys in a smaller range (especially some arbitrary small range) then you are putting a lot of trust in the operators of the HSM. hopefully there are good audit logs and people actually check the audit logs every time keys are generated to make sure the keys being generated aren't from a special range.

hey, at least it was easy to remember...

• Scott_Helme_ · 9 hours ago

  Can't lose your key if it's easy to remember!

In retrospect, having 100 trusted certificate authorities might have been excessive.

• tclancy · 12 hours ago

  In retrospect, not having a tiny competency check may have been an oversight.

  • oofbey · 2 hours ago

    Competence is difficult to check systematically. The problem is in the auditing process. The CA auditors check a much shorter list than an application that needs FIPS compliance. They just check that the CA is using a FIPS compliant key generator, and then basic things like following a logged, standard, multi person process. Anybody writing a system for a bank or the govt needs to have every line of code checked that it’s not doing something stupid with its crypto keys. That level of assurance would have caught this. But the process was defined too weakly.

    • tclancy · 2 hours ago

      Oh for sure. And at the time these were done, I'd guess cracking them was a lot more work. But still. 4.

      • knowinow · 1 hour ago

        This key 4 was from a new ceremony, they must have hit the highest security key just recently and wrapped around.

    • InsidiousWombat · 1 hour ago

      > They just check that the CA is using a FIPS compliant key generator, and then basic things like following a logged, standard, multi person process. FIPS-compliant key generator allowed key to be 4 ? No, they just didn't check thoroughly enough

• BirAdam · 12 hours ago

  or, perhaps you need more so that one being stupid doesn't effect so much.

  • caaqil · 11 hours ago

    > perhaps you need more so that one being stupid doesn't effect so much. So you create more surface area for vulnerability? We might already be at or close to the sweet spot. Security is hard.

    • Gabrys1 · 10 hours ago

      I would assume that the client would need to validate the certs against more than 1 trust path (for the comment to make sense)

  • epanchin · 11 hours ago

    Each CA is like a padlock locked in series with other padlocks. The weakest padlock is the only one that matters.

    • akerl_ · 10 hours ago

      In a world with CT logs and CAA records, is that actually the case?

      • zbentley · 9 hours ago

        Yes, because as we are seeing with this incident, plenty of things don’t check CT logs (and thus can still be hacked if CT isn’t published for bad certs), and CAA records are fundamentally advisory.

        • akerl_ · 8 hours ago

          You said "yes" but then answered "no"? Like it seems your disagreement is that you're pointing out there are systems today that trust WebPKI CAs but don't validate CT/SCTs. I'd agree that that's an issue and I want those systems to join the world where CT exists. As I said, in a world with CAA records and CT logs, this isn't an issue. If your CA wants to be in root trust stores, you have to put your certs in CT logs. CAA records are "advisory" in the sense that we mandate that root-trusted CAs obey CAA records and that they write cert issuance to CT logs, so a trusted CA that violates CAA records has to tell on itself in a very detectable way.

          • Scott_Helme_ · 7 hours ago

            I'm just not sure that I can see a world where all devices that rely on TLS/PKI are capable of CT SCT validation. Look at the issues we have around stale root stores still happening when we have validity periods of literally decades. Some of these clients out there simply won't ever be able to keep their trusted log store updated. Refs: Part 1: https://scotthelme.co.uk/impending-doom-root-ca-expiring-leg... Part 2: https://scotthelme.co.uk/complexities-chain-building-ca-infr... Part 3: https://scotthelme.co.uk/cross-signing-alternate-trust-paths... Part 4: https://scotthelme.co.uk/building-certificate-chains/ There's also the Let's Encrypt root migration that was postponed due to legacy devices: https://scotthelme.co.uk/lets-encrypt-postpone-isrg-root-tra...

            • akerl_ · 7 hours ago

              Similarly to vaccines, part of the magic is that you don't need "all", you just need "most". Right now CT support is rough for things outside of browsers; it's a pretty fair bet that if you pick up a command line tool or library, it is not checking SCTs, and that makes those workflows vulnerable. If we flipped that, and the average command line tool or library did validate by default, it would dramatically alter the risk calculus for attackers.

              • Scott_Helme_ · 7 hours ago

                For a scenario like this it doesn't help a given client what % of the ecosystem does SCT validation or not, all that matters is whether that client does it. I agree that more SCT validation across the ecosystem reduces the value of a mis-issued certificate to an attacker, but a lot of value still remains. If 90% of clients did SCT validation, an attacker can still succeed against the 10% that don't, there's no herd immunity.

          • lelandbatey · 7 hours ago

            You said: > In a world [where] CT logs and CAA records [exist] And we are in that world. CT logs and CAA records exist. They are around. What you are implying you said is: > In a world with CT logs and CAA records [and everyone/everything reads and validates them] We are not in that world, and we probably won't enter that world for the next 15 to 20 years because we are also in a world where folks expect networked devices to be able to communicate (mostly) securely without constantly communicating with a centralized set of clearinghouses arbitrating "what is secure". As we are discovering, our flawed systems may need broader rethinking, especially if we want secure communication (with all the epistemological ball of yarn that comes with it) without constant total centralization, which seems to be the approach-of-least-resistance lately.

• otabdeveloper4 · 12 hours ago

  Correct, we only need zero.

  • sebzim4500 · 11 hours ago

    Interesting, what would be the alternative? I mean specifically for the use case of the browser warning the user if there is a MITM between them and e.g. their bank.

    • embedding-shape · 11 hours ago

      The first time you go to the bank, you do the "trust exchange" (maybe scanning a QR? People sure love that) that verifies future connections, similar to how you acquire digital certificates today when you deal with the government online and similar. Edit: > Every time you access your bank on different devices normalizes it so that visiting a typo domain would show the dialog and many would just click through. Show the dialog? What dialog? There is no dialogues here, you cannot login to the bank unless you've verified the connection. You do it with your phone, then you can propagate the trust from there to other devices if needed.

      • whywhywhywhy · 11 hours ago

        Every time you access your bank on different devices normalizes it so that visiting a typo domain would show the dialog and many would just click through.

      • bluehatbrit · 11 hours ago

        Not all banks have physical locations, not everyone has the ability to get to the ones which do. You can't walk back from the existing expectation and ease of use we have today. It's just not going to happen, people will opt for the current path because it's easy and seems good enough to them. Most normal people don't even know this whole thing is going on. They don't know the difference between http and https other than "it's secure". If we want to improve on this, it has to come with the current level of ease of use at a minimum.

        • monsieurbanana · 10 hours ago

          > If we want to improve on this, it has to come with the current level of ease of use at a minimum. OR it has to be a situation catastrophic enough. I don't think this one is, but it's hard to be optimistic for the future

        • __MatrixMan__ · 9 hours ago

          Presumably you had some reason to trust the bank in the first place. Maybe a recommendation from somebody you know? Get the key from them. The current level of ease is not being achieved securely. If it's not possible to do so, we should walk it back.

          • pixl97 · 4 hours ago

            > Maybe a recommendation from somebody you know? This works great for something that are obvious. "Hey, this plumber was good and cheap". The problem is this completely and totally breaks down where things are obscure. When things turn from "reals" to "feels" the amount of manipulation a bad actor can perform is great.

            • __MatrixMan__ · 1 hour ago

              Agreed, but what would you propose we do about it? The current pki just washes its hands of the situation and restricts itself to only validating server names. Which probably made sense back when the people running the servers were also the authors of the content those servers served, but now it's a false sense of security. I'm trying to do more than that. If we validate claims made by the content as anchored in the user's trust graph, we don't have to care which server it came from--it's trusted for different reasons. Are you saying we shouldn't try just because it's hard?

              • pixl97 · 26 minutes ago

                It's not that we shouldn't try, it's that it gets depressing really quick because most of these efforts will fail for reasons of human behaviors. Trust doesn't scale, and the internet is something that operates at scale. All of the systems you're talking about have already existed and been tossed by the wayside. People could always have exchanged things like GPG keys, and yet here we are in a world in which we don't. The vast majority of your users do not want to manage trust. They want it to be taken care of. Almost no one gives two shits about a trust graph, except maybe nerds like you or me. Exposing the average person to these concepts doesn't increase trust. It just makes the world more confusing for them. This starts to delve into the monkeysphere and how many 'close' relationships people can form. If we have just a few things we trust it might work. Once you get over 15 or so it starts breaking down, and at 100 it's out of the question. Again, trust works on our evolutionary monkey brain that is used to a small tribe and systems requiring personal trust do not scale past that. Trust graphs are worthless when they are built on a house of security lies. It's easy to get kicked out of them, but nearly impossible to secure your systems to the point you can keep attackers from stealing them and pretending to be you.

        • chrisjj · 8 hours ago

          > If we want to improve on this, it has to come with the current level of ease of use at a minimum. We need to recognise that is impossible. Making getting into a bank account easier for the owner does so for everyone else too. And also makes it easier for the bank to ditch the secure fallback - a building. And ask why vendor-indepedent trust shops aren't sprouting in every locale. Answer: same security problems. We are well and truly doomed to living in a scammers' paradise.

          • venzaspa · 8 hours ago

            But my bank has no physical buildings whatsoever? And also, how is my bank supposed to safely generate and distribute a key that we can exchange securely if an organisations who's primary purpose to generate these keys couldn't even manage to do so. I have plenty of other services I use too. Suddenly my phone company, doctors, supermarket has to create and distribute keys for me to go around collecting so that I can securely talk to them and we have to do this in a way that stops anyone from swapping those keys out before they get to me. I mean the current system isn't perfect, but it's a Utopia compared to that.

            • chrisjj · 6 hours ago

              Yes. As I said, we're doomed.

      • u8080 · 9 hours ago

        Like...TOTP or FIDO2?

      • bee_rider · 8 hours ago

        That could be kind of interesting. It does seem to lean on the physical bank-front being trustworthy (no evil clerks). If we’re meeting physically to set up connections, we could even use something like a one-time-pad, and be information-theoretically secure. How many bits does your bank really need to send you securely anyway?

    • josephg · 11 hours ago

      Make dns secure and put certs / public keys in dns. Control over dns already gives you the right to get tls certs. It should just be connected directly. We don’t need CAs.

      • asksomeoneelse · 11 hours ago

        But how do you make DNS secure without CAs?

        • growse · 11 hours ago

          You just have one big one on the root. (Half joking).

          • asksomeoneelse · 10 hours ago

            Is that a reference to the Root Key of DNSSEC?

          • josephg · 9 hours ago

            Better to trust 1 organisation than trust 100.

      • shiandow · 11 hours ago

        Making DNS secure is not trivial though. In a way that's just moving the problem.

        • vesinisa · 11 hours ago

          Isn't this what DNSSEC solves?

          • muvlon · 10 hours ago

            DNSSEC deployment has been anything but trivial. This APNIC post from 2024 estimated that about 1% of DNS queries were being validated via DNSSEC: https://blog.apnic.net/2024/05/28/calling-time-on-dnssec/ . That's more than 25 years after the IETF first standardized DNSSEC.

            • giantrobot · 9 hours ago

              2107: The year of IPv6, DNSSEC, and The Linux Desktop.

            • tptacek · 5 hours ago

              I track DNSSEC adoption across the Tranco research list of top domains: https://dnssecmenot.fly.dev/

          • shiandow · 10 hours ago

            It should, though it achieves that goal by a chain of certificates not unlike CAs just tied to DNS. Like I said, it essentially just moves the problem. Which might still be better, if only because it limits the surface area.

          • SAI_Peregrinus · 3 hours ago

            No, DNSSEC tries to solve one part of the security of DNS (security of records between resolvers) and does nothing for the security between the user and their DNS server. And doesn't even do a good job at the bit it tries to solve.

        • josephg · 9 hours ago

          A secure DNS is valuable whether or not we get rid of CAs, since CAs rely on DNS queries to verify domain ownership. And if we have a secure DNS, then CAs are completely redundant. I'd much rather trust a handful of domain registrars than a handful of domain registrars, plus 100 random questionably competent organisations all over the world. Especially when any one of them can generate keys for every domain on the planet.

          • darkwater · 9 hours ago

            It seems a nice thing to have indeed, but domain registrars have already a history to cede control and bend to $GOVERNMENT. We have countries already trying to MiTM all their citizens by mandating CAs injection in every device under their jurisdiction, what would be the defense against this threat in a world where registrars hold the keys to TLS encryption for the web on the TLDs they own as root? I'm not that expert in this field so I really would like to learn if it's possible.

            • shiandow · 7 hours ago

              Not that it isn't an interesting question but what's preventing them now? You can get a certificate for any domain you control. It is just really obvious if you abuse that power. Which shouldn't really change with DNSSEC I think.

            • GoblinSlayer · 5 hours ago

              Government can compel servers in its jurisdiction to use a mandated CA. DNS would nicely compartmentalize it to the zone controlled by that government as opposed to the CA system where every random eMudhra CA has infinite reach.

          • tptacek · 5 hours ago

            "Secure" doesn't mean anything. A closer (though still imprecise) descriptor is the CIA triad --- confidentiality, integrity, and availability. A confidential DNS has real value. Without it, ISPs actively monitor customers and sell analytics about usage upstream; there are other problems, but virtually every US user faces that one. It's hard to imagine wanting to trade confidentiality off. A high-integrity DNS is a more dubious proposition. You'd obviously rather have integrity than not, ceteris paribus, but we have effective mitigating controls for DNS forgery, so you might imagine trading it off. Availability also sounds like a no-brainer, until you remember that almost every other component of the networking stack also has availability issues. You might not want to trade anything at all to get a more highly-available DNS, if you're going to be just as susceptible to DOS attacks because the current DDoS meta is application-layer anyways. There are a bunch of different approaches to securing the DNS, and they make different tradeoffs. DoH focuses on confidentiality, and has bank-shot benefits to integrity (because it breaks forgery attacks from on-path attackers). Confidentiality is its primary purpose and it trades basically everything else off. DNSSEC makes the opposite tradeoffs: it's exclusively for integrity, provides no confidentiality, and highly prioritizes availability (that's why it has elaborate authenticated denial systems). Depending on what you want to achieve, "secure" DNS can make sense and actually get deployed and used, as DoH is, or it can make almost no sense and see almost no uptake.

            • GoblinSlayer · 4 hours ago

              Security provided by CAs is TLS Web Server Authentication (1.3.6.1.5.5.7.3.1). CAs too are exclusively for integrity, provide no confidentiality and don't prioritize availability.

            • InsidiousWombat · 1 hour ago

              DoH was mostly forced by few companies, it doesn't solve any real problems because the places that would like to spy on your DNS can just block few DoH providers that are used by default. Like, it's nice to have that random MITM can't see your DNS but if ISP wants it, they can just block it.

              • tptacek · 38 minutes ago

                This mostly describes the difference between DoT (which is trivial to block) and DoH (which is deliberately harder to block). Are you aware of any ISP blocking DoH?

      • braiamp · 11 hours ago

        That still requires trust stores to be distributed to the clients and some clients don't need that baggage on the dns. Also, dns is by default insecure, dnssec is a configuration nightmare

        • otabdeveloper4 · 11 hours ago

          > That still requires trust stores to be distributed to the clients There's no such thing as a "trust store" in practice. To be "trusted" you just need to demonstrate you have control over a DNS TXT record, and we don't need a special "store" or intermediary for that. Certificate authorities are an artifact of a bygone time and an entirely different model of internet infrastructure.

          • woodruffw · 10 hours ago

            The trust store refers to the collection of parties that ultimately verify the signed claim of your control over a domain, which might have taken the form of a DNS TXT challenge (or any other number of challenges). You still need it, because there would otherwise be a basic circularity when determining whether to trust a connection. (Moving the PKI into DNS does not remove it, it just moves it around.)

            • josephg · 9 hours ago

              > (Moving the PKI into DNS does not remove it, it just moves it around.) Better in DNS though. You could have certs mirror the structure of DNS. Install a single root signing key for the top level DNS registry on client devices. Then you can query that to find out the signing keys for .com, .net, .au, etc. And continue from there. One big problem with the CA system is that any CA - even the most incompetent - can sign certificates for any domain. If trust was moved to DNS, there would only be a single organisation that could forge certificates for any given domain. Using 7 as a private key? What a joke. The CAs aren't worthy of trust. We already have to trust the domain name registrars anyway, since CAs query them to figure out if you own a domain before signing a certificate. In total, removing CAs would not only simplify the system. But it would allow scoped signatures. And we'd end up needing to trust far fewer actors to have a secure internet.

              • woodruffw · 9 hours ago

                Why would I trust my domain registrar to maintain PKI? They don’t seem inherently better suited for it, and in various ways are more susceptible to meddling for e.g. ccTLDs. I think the idea that CAs are uniquely incompetent is probably incorrect.

              • itintheory · 7 hours ago

                > One big problem with the CA system is that any CA - even the most incompetent - can sign certificates for any domain. Is this not what CAA records were intended to solve? It seems like in practice they're only used by CAs to decide if they can sign a cert for a given domain, but it seems like a client could use it to determine if a server cert was from the correct CA.

                • Scott_Helme_ · 6 hours ago

                  CAA records are only checked by the CA at the point of issuance, not by the client later. If they did, there are problems to consider: 1. A domain would have to keep the CAA records for a CA in place to keep using the certificates from that CA, but that means continuing to authorise new issuance too. Right now, I can remove the `letsencrypt.org` value to stop them from issuing new certificates for me, but I can keep using any certificates I have until they expire. 2. The client would need to have support for this new DNS RR type. 3. The client would need a secure way to query DNS, which is not widespread.

                • parasyte · 5 hours ago

                  CAA records were never intended to be used by clients (Relying Parties), the RFC specifically prohibits them from using the CAA record to determine if the issuer was allowed to have issued the cert. I infer from the wording around there and elsewhere in the RFC that this is mostly because someone could switch issuers but still serve valid certs from the prior issuer, and there's no way an RP would have a historical record of which CAA values were in effect when to be able to determine if the issuance was valid or not.

      • three14 · 10 hours ago

        tptacek argues that DNSSEC is too flawed to actually do this: https://sockpuppet.org/blog/2015/01/15/against-dnssec/

        • ranger_danger · 9 hours ago

          I have seen their comments across this site for a very long time. Unfortunately I do not respect or put any stock in what they say, because of how unbelievably rude and dogmatic they are, and always have been. "As a rule, strong feelings about issues do not emerge from deep understanding."

          • icedchai · 9 hours ago

            Fortunately, you can investigate this stuff for yourself and develop your own opinion. IMHO, DNSSEC doesn't actually "secure" much of anything. It can be disabled trivially by anyone operating a resolver. As I mentioned elsewhere, none of the major sites (FAANG) use it. What does that tell you?

            • ranger_danger · 8 hours ago

              Yes I'm not saying DNSSEC is or isn't good. I think a multi-layered approach would be best for redundancy.

              • tptacek · 5 hours ago

                Wait. Hold on. It is either good or it isn't. If it's good, we should be using it (maybe in a layer cake, maybe a la carte). If it's bad, it should not be used anywhere.

            • GoblinSlayer · 5 hours ago

              Then propose a censorship resistant protocol.

        • josephg · 9 hours ago

          He's right. DNSSEC is a mess, and we should fix that. We should fix that whether or not we delete CAs because CAs rely on DNS to verify that you own your domain. If DNS isn't secure, neither are CAs. In the long run, fixing bugs is almost always better than working around them. DNSSEC is a bug. So are CAs. We can and should fix both.

          • chimeracoder · 8 hours ago

            His argument is that DNSSEC is fundamentally broken, and you can't fix it via incremental improvements. He is not arguing that DNS security cannot or should not be improved; in fact, he explicitly states the exact opposite.

            • tptacek · 7 hours ago

              I don't have an objection to doing a different DNS security system, and in fact I like DoH a lot. You can do DoH all the way back from authority servers if you like, and that addresses domain validation without boiling the ocean with 1990s cryptography the way DNSSEC tries to do.

        • GoblinSlayer · 8 hours ago

          ECDSA was added to DNSSEC 3 years before that blog post in april 2012 (rfc6605).

          • tptacek · 7 hours ago

            And then for the next 10 years APNIC wrote posts about how actually using ECDSA broke resolvers. It's going to settle into general usability at just about the time cryptographers deprecate ECC entirely!

            • GoblinSlayer · 4 hours ago

              They can be deprecated for confidentiality purpose, but DNSSEC doesn't use confidentiality.

              • tptacek · 4 hours ago

                You're trying to argue that DNSSEC isn't exposed to HNDL attacks, so can use signature algorithms that CRQCs break. But that's neither here nor there: the approaching cryptographic research consensus is going to deprecate all ECC (and all RSA); yes, signatures aren't the priority for immediate removal, but nobody is recommending ECC signatures. The point is all this is happening just as ECC in DNSSEC may finally be stabilizing.

      • jayd16 · 6 hours ago

        So like, the cable company now controls the certs for most consumers? shudders

        • tptacek · 5 hours ago

          Yes. This is basically the DANE pitch. Subtextually, supporters argue that two big shifts will immediately happen: (1) We're also rearchitect the way all DNS is done so that every lookup from every phone and laptop is fully recursive, so that everyone validates DNSSEC chains instead of relying on the AD bit that cable company DPI devices switch off. (2) People will lose all attachment to their current domain names and simply migrate to TLDs that have policies they agree with, rather than .COM and .IO, which are under the demonstrated de facto control of the NATO IC.

      • 1vuio0pswjnm7 · 2 hours ago

        Perhaps browser fans will check the source code to verify that their browsers or operating systems using DoH ("Private DNS" or whatever they call it) check the certificates presented by DoH providers against CT logs The dependency of "accetpable", i.e., not self-signed, TLS certificates on DNS, i.e., a domain name in a TLD created by "ICANN", is funny when one considers that the traffic to authoritative nameservers is still unencrypted. It stands to reason that DoH and other "accepted" forms of "DNS security" are only to stop ISPs from stealing Silicon Valley's surveillance advertising racket. Neither is "DNSSEC" providing the user with any real "security"; it's protecting a monopoly. The "security" is against anyone who might try to use domains names not blessed by "ICANN" Indeed there's no need for CAs just as there is no need for ICANN

    • lukewarm707 · 10 hours ago

      For me the complexity of anything connected to public internet is now over what I can secure. I genuinely can't say I secured a PC, when it needs to phone home for a configuration at this point. We are most likely over the line for needing to airgap a personal PC.

    • Ohentis · 10 hours ago

      I quite like how i2p does it. Public keys are part of addresses, human readable names are user defined and shared but static.

    • __MatrixMan__ · 9 hours ago

      Stop validating names, start signing content. Scan a QR code when you open the bank account, add it to the keys you trust as authorities on financial things. When your browser runs accross data tagged "financial" it checks the trusted keys in that domain and renders it only if it was signed by a key that is trusted in that domain (the domains prevent your bank from abusing that trust by causing you to see political ads or some such). If you ever see something that you suspect is malicious, you untrust whichever key signed it and take it up with whoever asked you to trust their key. If you verify it's malicious, gossip the bad keys to your friends so they can distrust also.

      • diarrhea · 9 hours ago

        That solves integrity but not confidentiality. It does not protect against MITM.

        • __MatrixMan__ · 6 hours ago

          It does protect against MITM. A man in the middle can pass the content along or they can refuse to do so but they can't forge a signature. If confidentiality is required, well you already have a public key that you're using to verify content, then the content can say: "reach us at 196.7.3.45" and then you encrypt communications to that address with the same public key you use for validating content. Or maybe they'll specify, in the content, that you should use a different key for that, but either way the chain of trust is preserved. This would of course all be protocol level stuff, handled by the browser, not a human. They just have to maintain a list of keys they trust.

          • pixl97 · 5 hours ago

            >It does protect against MITM. A man in the middle can pass the content along or they can refuse to do so but they can't forge a signature. Um, I don't think you understand this problem near as good as you think you do? >A man in the middle can pass the content along or they can refuse to do so but they can't forge a signature. So where do you get this signature exactly so they don't MITM you the first time? > well you already have a public key that you're using to verify content, Who what where when why? Where did I get this public key? Who provided it? Is this my public key or their public key? How do I trust it? >They just have to maintain a list of keys they trust. So you've solved exactly nothing.

            • __MatrixMan__ · 3 hours ago

              > So where do you get this signature exactly so they don't MITM you the first time? That's always going to depend on why you're trusting them in the first place. If somebody handed you a business card the key should be on the card. If it was a recommendation from a friend you should use whatever key they use. If you're expecting to just type random IP addresses into your browser until you hit something and then be told that you should trust what you see... Well that's not in the cards because you shouldn't trust what you see in that scenario. We already don't type in URL's, we send links to each other. It really wouldn't be so difficult to change the UX so that when we delegate trust in that way it's public keys that are being trafficked instead. > So you've solved exactly nothing. Removing untrustworthy third parties and the need to revoke trust in them on the user's behalf is the solution. Users don't know what a CA is or how they should evaluate its trustworthiness. We need a system where they're instead evaluating the trustworthiness of things they understand, like other people. When they get burned they should have the ability to immediately revoke trust in whoever burned them. This thing we're doing where we expect them to wait around while some nerd pushes the updated list to them and to blindly trust that that nerd isn't curating that list with ulterior motives is totally bonkers.

      • chrisjj · 9 hours ago

        > When your browser runs accross data tagged "financial" it checks the trusted keys 'Cos no scammer would ever be so sneaky as to leave off the tag... /s > it up with whoever asked you to trust their key 'Cos no scammer would ever pretend to be that person... /s

        • __MatrixMan__ · 5 hours ago

          Well if you find unsigned content, you should probably just not show it to the user at all. As for tagging... We could use something like color to indicate domain. So if your well trusted plumber is giving financial advice maybe it shows up, but not in the "financial" color? Idk it seems like a solvable UX problem.

          • chrisjj · 4 hours ago

            Anything relying on user recognising some tag colour seems like an unsolvable UX problem to me.

      • lokar · 7 hours ago

        So, have each domain run its own root CA (scoped to the domain), and have users trust them one at a time when they have cause to interact with them?

        • __MatrixMan__ · 6 hours ago

          Yes to smaller scopes, with keys passed of during interactions that organically arise, preferably face to face, but maybe not always. No to making domain names be the thing that's being trusted. We should push this into the application layer, so rather than scoping the keys to domain names let's scope them to something relevant to whatever the user authentically trusts. Something to do with their intentions for using this app. Because most users don't even know what a CA is, but they understand that most of the time they'll need to trust somebody. So let the app decide what that's going to mean because those dynamics are always going to depend on the situation. So the user (or OS) would pass an object at runtime to indicate which keys they want the process to trust for this session. Typically it'll be another human or a company, not a server name, but I suppose it could also be a device. Something that can guard a key pair and be identified by its ability to produce valid signatures. That way you're just not relying on globally unique human friendly names at all. Doing so puts you in a cursed corner of zooko's triangle. Users can assign a pet name for whoever they're trusting at the time they're configuring that trust since that's the moment they're best able to scrutinize the trustworthiness of this individual or company or device. Not later on when they're installing software updates and have no context for the decisions they're making. This way your application can work over alternative transport layers: carrier pigeons, lobbing an SD card over the border wall, whatever, and doesn't break just because the other party has a different name on the new transport. Relevant for when the internet goes down and you still want it to work with other hosts on the LAN or the local mesh or whatever.

        • xp84 · 2 hours ago

          In the modern day that's arguably fine, most users only interact with a few actual companies online (though many of them haphazardly use dozens of domains and dozens of unrelated certificates). The only 'real' problem though with this approach is it requires a trusted path for the bank to get the "JP Morgan Chase root CA cert" into my hands (like a QR code they hand me physically). Serving it up over http: or over an unvalidated SSL context at first use seems to defeat the purpose since I could be being MitMd the first time I hit chase.com. In practice most people would be angry that they couldn't use the bank's website if their new phone or PC didn't have the certificate yet, and if places start advertising their cert QRs everywhere, it will normalize the idea that you add a CA certs all the time, leading to CA cert QRs being on billboards and bus stops and eventually fraudulent ones would be snuck in that way.

      • tptacek · 4 hours ago

        You are describing key pinning. You have invented QR-code-based key pinning.

        • __MatrixMan__ · 2 hours ago

          My intent is to provide users with the ability to revoke trust if in a source of information if they realize it is malicious, and to gossip that revocation among their peers. RFC 7469 appears to be more geared towards service providers attempting to protect their users. Laudable, but we're still assuming we know something about who the user deems trustworthy, which I think is presumptuous. It's one thing to suss out cases where somebody is lying about their name, that's pretty cut and dry. But ever since web2.0 there's no longer much of an association between the hostname and the trustworthiness of the content. If we instead provided a means of validating the content's provenance it wouldn't much matter which host was involved in its delivery.

    • rocqua · 8 hours ago

      In practice we have Certificate Transparency Logs that let us check whether a CA has issued a certificate for a domain you control without you asking them to. This really reduces how much we need to trust the CAs, because we can detect any untrustworthy actions after the fact. Now detecting after it happened is too late. But it does mean that mistakes or malicious actions are much more expensive to a CA. In practice this has really reduced the risk, especially to those whom aren't interesting enough to burn a valuable capacity on. Which is the vast majority of people. Though perhaps if you count TLS traffic weighted by importance that count is different.

      • zaptheimpaler · 4 hours ago

        I think someone explained that only browsers and maybe some applications check the CT logs in the first place. Most applications don't, so someone can impersonate any URL an application connects to and the attacker doesn't have to publish to a CT log.

      • krackers · 2 hours ago

        How would certificate transparency logs have helped here though? If the private key leaks and anyone can mint a valid certificate that gets accepted, those logs aren't going to reveal it.

        • Scott_Helme_ · 12 minutes ago

          If an attacker takes the CA key and issues a valid certificate, but does not log it to CT, mainstream browser clients will not accept it. When a certificate is logged to CT, you get back an SCT, and the issuing CA will normally include that in the certificate. The browser will then accept this certificate as it has the correct SCTs (there are always at least two). The attacker issued certificate hasn't been logged, so can't have an SCT, so won't be accepted by the browser. We can't say that no certificates have been issued by an attacker. We can say that no certificates have been issued by an attacker that a modern browser client would accept.

    • InsidiousWombat · 1 hour ago

      Zero would be hard but we could have signed CA cert put into DNS record and from that the org itself can create the certs for that domain. Then at the very least one of the CAs leaking isn't automatic compromise, as leaking the key still doesn't make you insecure, that would also require intercepting and changing DNS records

• m3kw9 · 9 hours ago

  How are they trusted if they can just generate any key they like in their back room?

  • bigfishrunning · 9 hours ago

    That's the thing about trust -- in order to get into the root key store for windows, chrome, linux, etc they were trusted not to do that. they violated that trust.

  • swores · 9 hours ago

    "Trusted" means how people treated them, it doesn't mean the extent to which they deserved trust.

    • chrisjj · 9 hours ago

      This. Good: trustworthy Often bad: trusted

  • RALaBarge · 7 hours ago

    There are chains of trust which are established for SSL certs, centered around the idea that if we can believe Mozilla et al. that the server responding is who they claim they are. Up until now, that is good enough to establish the trust needed to accept the 2 keys in the connection are connected, and then a secure connection could be established to the host server. Mozilla et al. could claim trust for Microsoft and then MS would be an intermediary for Mozilla. There is a long history of sketchy intermediaries like GoDaddy and others that root certificate providers like Mozilla would block at times due to abuse. This leads to users, usually ones that are not that technically savvy, seeing 'insecure' warnings on a page that actually offers HTTPS. ..but since the root servers no longer accept them as trusted, everything that GoDaddy supported, and every intermediary (child) cert which they approved would no longer be trusted and you would see a non-secure warning.

    • BubbleRings · 1 hour ago

      Which leads to users that are not that technically savvy learning to always click "Load it anyway!" and not think about it, even in financial management firms. Ask me how I know...

• Scott_Helme_ · 9 hours ago

  That really depends how you define "trusted certificate authorities". If you're talking about root certs then it's almost 350 currently trusted by at least one major operator. If you include intermediates, you're talking more than 3,600!

• londons_explore · 6 hours ago

  Certificate transparency requirements effectively take the power away from certificate authorities. They instead place the burden on individual webmasters again to actually verify that nobody has been falsely issuing for their domains. There are also some new vulnerabilities about CT logs who might be malicious.

  • Scott_Helme_ · 6 hours ago

    I agree this is a burden placed on domain owners, but who else can know if a certificate for my domain 'scotthelme.co.uk' was supposed to be issued other than me, the owner of 'scotthelme.co.uk'? Do you have any info on the malicious CT logs? I'd be interested to read something on that!

    • jasomill · 2 hours ago

      Possibly CT logs revealing information about domain name and IP address spaces, or other information through implementation details, that might not otherwise be public.

    • londons_explore · 2 hours ago

      CT logs stapling proof, but then never actually including that cert in the log.

      • Scott_Helme_ · 16 minutes ago

        But that's detectable and what we have MMD for, a log operator that does this can be found out.

eMudhra explanation : https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c11 Kind of hard to understand how a bug would have generated a second pair of key so bad, and how would human would have ended up using them. Never attribute to malice what can be attributed to stupidity, but sometimes you wonder.

• sobkas · 12 hours ago

  Any sufficiently advanced malice is indistinguishable from stupidity.

  • SAI_Peregrinus · 10 hours ago

    Never assume stupidity for what can adequately be explained by profitable malice. Never assume unprofitable malice for what can adequately be explained by stupidity.

  • RetroTechie · 10 hours ago

    The reverse is also true: Any sufficiently advanced stupidity is indistinguishable from malice.

• dist-epoch · 12 hours ago

  And an equal amount of stupidity in the whole ecosystem, apparently there are no "unit-tests" for trivial failure modes, like 8 bits of entropy, before adding a CA to the trusted list. What next, googling the public key finds a 2 year old pastebin with the private key?

  • silvestrov · 12 hours ago

    Even just a human glancing at the private key should do it. Don't they even test script changes on a staging server? How could there be such a script error in production that was not caught in testing/staging? This sounds very like a fundamental organization error, not an innocent whoopsie. It should lead to a permanent ban of the organization.

    • karel-3d · 11 hours ago

      Well humans should NOT glance at private keys! That is the whole point of HSMs!

      • consp · 11 hours ago

        I'm still flabergasted the HSM allowed this key to be generated. Then again, I've worked with HSMs, there are always many mistakes in the firmware, usually less severe though.

        • karel-3d · 10 hours ago

          It seems there was some issue with the ceremony with some wrapping/unwrapping scripts. They explicitly said they wanted to export the keys from the HSM; the keys were intended for USA/Europe (for sovereignty reasons) but were never actually used. It seems the HSM itself worked, their ceremony around wrapping/unwrapping didn't.

          • consp · 10 hours ago

            Makes more sense. I'm sure the Utimaco's used (haven't worked with one) also allow all slots to contain the same name or keys with the same name and the device always selecting the first one if you only provide the name and not the slot ID or token identifier. Don't ask how I found out the hard way.

            • karel-3d · 10 hours ago

              Yes there was something about key names in the write-up (which I cannot find now) oh here https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c11

      • silvestrov · 9 hours ago

        But eMudhra wanted to (was required to) export the keys. So using an HSM in the first place is a security theater. Or that this HSM isn't really an HSM since it allows for export of the private key. The trusted enclave in a MacBook is more secure than this HSM.

        • xvector · 9 hours ago

          Most HSMs used by CAs are pretty trash tbh.

        • karel-3d · 9 hours ago

          There is something called "wrapping/unwrapping" - you encrypt the key, move it to the other HSM, and it unwraps it. So what moves around is encrypted. I don't know enough about HSMs to tell you if it's done using symetric or asymetric crypto, but no the key should not move around in the plaintext. The write-up from eMudhra mentions something about wrapping/unwrapping ceremony failing.

• amoss · 10 hours ago

  Given the usefulness of this to anybody who knew about it between the ceremony and now - assume malice.

  • palata · 1 hour ago

    If I was to leak a private key for malicious use, I would leak a good key that only I know.

https://xkcd.com/221/

• Qision · 12 hours ago

  Literally the tutorial they used

• Scott_Helme_ · 9 hours ago

  I think this might be proprietary source code, did you get permission to share this?..

Could HN start feeding submissions through Pangram so us human readers don't have to suffer with reading the nauseating AI-prose?

• jadamson · 12 hours ago

  It reads fine to me. If it is AI, I'd like to know which one so I can use it too.

  • TazeTSchnitzel · 12 hours ago

    I think it's a hybrid text. The section about how elliptic curve keys work is particularly aggravating. Unfortunately it's also the only section that gave me new information.

  • stratos123 · 12 hours ago

    It reads okay to me too, yet Pangram says it's mixed - the "What a private key actually is" section is AI, so is 'Why "it cannot be contained"' and some others.

  • karel-3d · 11 hours ago

    It has parts that are human written and parts that are generated. I couldn't tell at first, but now seeing it in Pangram, it's kind of clear.

    • Scott_Helme_ · 9 hours ago

      It was all written by me and then I had final pass/tweak with Claude.

      • queenkjuul · 5 hours ago

        So parts were generated... Unless you're saying Claude just went "lgtm, ship it"

        • Scott_Helme_ · 5 hours ago

          Fair enough, call it a mix if you like. I'm happy to stand behind every claim in my post, and if something's wrong, tell me and I'll fix it.

• swiftcoder · 12 hours ago

  Count yourself lucky - compared to Claude's prose, this is fairly readable.

"Importantly, none of the affected intermediate CAs were deployed in production, with the exception of one. Furthermore, none of these intermediate CAs, including the one deployed in production, have issued any end-entity certificates. This has been confirmed through our internal analysis and corroborated by publicly available Certificate Transparency records on crt.sh." This statement is more discerning than anything else. It raises question about why this certificates exists in first place if not deployed since 2025. Is it deliberate attempt to do compromising of system? In recent times, Indian Government is becoming quite rogue and don't follow the said procedures. So, it might be related to that as well.

This is not a good write-up. Yesterday's discussion did contain better data, less self-Promotion and actually asked why it happened, instead of just saying "number small".

I will say, there’s something almost reassuring when something goes boom that badly and the story is really silly. It’s like [horrific human-caused disaster] but it’s because of a Three Stooges episode.

• skinfaxi · 10 hours ago

  The more cynical among us think that is part of it. Remember when Bush Jr. became a bumbling idiot for a few years?

I wouldn't be surprised if we learn in the near future, that this was deliberate.

• yoavm · 12 hours ago

  Why would it be?

  • sscaryterry · 12 hours ago

    Do I honestly need to enumerate what you can do if you have the keys for a CA?

    • swiftcoder · 12 hours ago

      Yeah, but if you are the one issuing the CA's private key, you already , by definition, have the keys to that CA. You don't need to make stupid keys in that scenario

      • sscaryterry · 11 hours ago

        In case it wasn't obvious. The NSA and many other organisation would find this extremely useful.

        • yoavm · 11 hours ago

          But it doesn't make sense. Sure they'd find it extremely useful to have the key, but it doesn't need to be a weak key for it to be useful. If anything, it makes it less useful to them. If I'm going to ask you or force you to share your password with me, there's no reason I'd ask you to set the password to 4. We might as well agree that the password would be D#!(s9fj0jalk!4af095SF42 , and it's much more likely our collaboration will continue working for years. "Never attribute to malice that which is adequately explained by stupidity"