logo CoolAJ86

CSR, My Old Friend

Published 2018-11-11

Part 1

This is the story of how I figured out a stupid-simple solution to generating (and potentially parsing) ASN.1 and X.509 (for certificate requests in my case).

This is not a quick story. The, ultimately simple, discovery came after much (read: years of) toil, which was riddled with emotional turmoil and frustration. Hence, I'm not diving straight into the solution, but rather recount the "joy" of the journey.

(that said, feel free to skip ahead to the goodies - I won't know, so my feelings won't get hurt)

So...

how did I come to be one of the few and unfortunate people that can understand and explain ASN.1?

TL;DR

I was frustrated with how difficult it was to generate a CSR in JavaScript and so I went down the rabbit hole and ended up creating two great (in my opinion) solutions:

If you want more of the details of what I actually learned, that's available now too:

The Backstory

(a.k.a How I got this emotional baggage)

I'm working on an in-home personal cloud, which I've always wanted to be a consumer device. Obviously I want it to be safe for the non-technical populace that I envision will one day be using it, so automating all of the security and presenting it in a non-technical way has always been a top priority.

At the time I started I had only a faint idea of how I might solve the problem of automating https. I was moving forward with development despite a seemingly insurmountable task up ahead... and then Mozilla + EFF announced Let's Encrypt. The timing literally could not have been more perfect.

The IGG had actually prototyped some of the ACME protocol in node (for which I've been a long-time developer and was already using for this project) before moving the client to python and the server to Go.

Someone had gotten a rudemantary demo working, which I forked and morphed and rebuilt as letsencrypt.js, which later became greenlock.js (now complete with Let's Encrypt v2 support) - and I even ported it to the browser (w00t!).

The struggle has been real and I can see why the IGG and EFF moved away from JavaScript. The crypto story is terrible.

The Usual Suspects

I've only been able to do as much as I've done by mixing a hodge-podge of (literally) megabytes of JavaScript from multiple suites at a time including forge.js, pki.js, ASN1.js, elliptic.js, etc. You know, the usual suspects (excepting jsrsasign, which I somehow managed to avoid).

Of course, none of them are complete enough to do all that needs to be done. They've all been built at different times, so they have different, incompatible ways of handling binary data, requiring you to write special one-off converters as you pass data between them to get a complete flow.

You have to use a different set in node than in the browser. And each browser has a varying level of support for the necessary features.

Fortunately, in recent months it has become possible to do pretty much everything in native node crypto and native webcrypto (in almost all browsers), except for one thing...

CSR, My old friend

Certificate Signing Requests (or CSRs) are a doozy. They're kind of like JWK+JWT of the dark ages.

The underlying format that is written in is called ASN.1 which is like the OG* XML (which is like the JSON of the dark ages, for those unfamiliar with XML). But layered on top of that is a standard called X.509, which is pretty much the OG XLST (think json-schema.org of the dark ages).

* that's Original Gangster for those not in the know

Summarized in a different way (and yet still step further down the rabbit hole):

  • PEM is simply a base64 encoded DER file (with a plain-text header and footer)
  • DER is the binary format of ASN.1, more or less a dark-ages BSON
    • (you colud say DER is to UTF-8 as ASN.1 is to Unicode)
  • ASN.1 would then be a sort of dark-ages object notation for anything
  • X.509, then, is a dark-ages schema that uses ASN.1 to describe a specefic, concrete thing

So you see, there is no new thing under the sun. What's old is inevitably new again.

The General Problem... It's complicated

As programmers we like to make this overly complicated - but our hearts are in the right place. We try to do it for a good reason.

See, when we do things the "stupid" way we look... stupid. If we do it the dumb way it's easy for someone to point out "hey, that's dumb!" and, inevitably, excuse ourselves with "oh, this is just spaghetti code because... legacy/tech debt/learning" to to explain it away with less guilt.

To compensate we get clever. Too clever. We over-engineer. We design things to be so uber modular as if to solve all possible problems rather than just the problem at hand. (xkcd#947, anyone?)

Imagine that the JSON spec itself include everything on json-schema.org - that's near the level of entanglement that ASN.1 seems to have with its schemas, with every thing defined by some strange level of section, subsection, and paragraph header naming scheme that looks something like the dewey decimal system for object notation.

I mean, how the heck does a type end up with `1.2.840.113549.1.1.1' as its identifier? How many types were there supposed to be in this system? Oh, all of them.

Arriving at an easier approach

Complicated standards - especially complicated binary standards - especially complicated binary standards that only really ever had a single use case (and were then abondened), as you might imagine, require complicated code to parse and generate. (these are "solved" problems, you see, that we needn't bother ourselves with any longer, generally speaking)

When we loop back around to the issue of JavaScript in particular not having a good story (no standards) for how to transfer binary-ish things like bigints and, well, straight binary, you can start to see why multiple libraries and custom conversions would be needed to complete a flow for a single task.

For so long I've wanted to find a less capable, stupider, but cleaner solution but everything seemed so complicated I thought that I was simply subject to the powers that be.

And then, finally, I was shown the light. Someone commented on my Let's Encrypt v2 Step by Step guide with a cobbled together, half-baked solution so stupid that I immediately believed that it was not just a solution that could work, but probably the way that anyone who has ever done this probably should have done it to begin with - instead of actually understanding ASN.1, just make a template to copy the bits that stay the same and paste in the values that are different.

Many thanks to the man who showed me how easy it could be to work backwards.

Reverse Engineering

There's an old wise saying of our dark arts:

Remember, a few hours of trial and error can save you several minutes of looking at the README - @iamdeveloper

But equally true we must also concede to the counter point:

Generally a few hours of trial and error are necessary before the README makes any sense! - @frogfather

In my search to decompose the mystical CSR I quickly came across the golden nugget, https://lapo.it/asn1js/. However, I couldn't yet make sense of it. I needed to dive a little further.

Rather than approaching the problem from the point of the "general solution", I figured that if I could just generate a known-good CSR for just one key type and just one profile - say either 2048-bit RSA or a P-256 ECDSA - I could make several variations changing only exactly one thing at a time and learn how a CSR was constructed:

  • which bits always stayed the same
  • which bits represented the domain names
  • which bits indicated length
  • which bits indicated the number of domains

With a little help from @mathwhiz1212 I was able to create the necessary CSR permutations to do just that.

The basic script to generate a keypair and CSR is as follows:

# generate keypair
openssl genrsa -out ./privkey-rsa-2048.pem 2048
openssl rsa -in ./privkey-rsa-2048.pem -pubout -out ./pubkey-rsa-2048.pem

# generate csr (with inline config file)
openssl req -key ./privkey-rsa-2048.pem -new -nodes \
  -config <(printf "[req]
prompt = no
req_extensions = req_ext
distinguished_name = dn

[ dn ]
CN = example.com

[ req_ext ]
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = example.com
DNS.2 = www.example.com") \
  -out example.com-www-rsa-2048.csr
node convert-to-der.js example.com-www-rsa-2048.csr

These were my target variations:

And I was able to use Hex Fiend (try DHEX on linux) to diff the files to see what was changing. After a few rounds of opening two files individually and selecting "Compare" from the File menu to actually start the diff, I was able to "get my bearings", if you will, and the ASN.1 Debugger went from looking like gibberish to making perfect sense. Yay!

Voila! I created my own CSR Game Genie*!

* this is exactly how the Game Genie worked (and how all reverse engineering works): targeting and tracking changes

Up Next: ASN.1 and X.509 Explained

Update: The conclusion is here:

Here we are at the meat and potatoes, but I'm tired and want to head to bed.

But the conclusion is coming, I promise!

Update (Tuesday Nov 13): There are just a few kinks I need to work out before I do the next post.

Update (Thursday Nov 15): I'm going to need this next weekend to finish this. I ran into some stuff that needs further exploration.

Update (Tuesday Nov 20): Sooooo.... close

Update (Sunday Nov 25): Finally working on the article today.

The result of my learning became these great (in my opinion) solutions:

  • ECDSA-CSR.js - Generate a CSR with EC keys
  • Eckles.js (w/ CLI) - Generate EC keys, convert between JWK, PEM, and SSH, and generate CSRs
  • RSA-CSR.js - Generate a CSR with RSA keys
  • Rasha.js (w/ CLI) - A lightweight VanillaJS library to generate RSA keys, convert between JWK, PEM, and SSH, and generate CSRs

By AJ ONeal

If you loved this and want more like it, sign up!


Did I make your day?
Buy me a coffeeBuy me a coffee  

(you can learn about the bigger picture I'm working towards on my patreon page )