There's more than one way to skin a cat, and there's also more than one way to do a lot of other things - like store and cache your git credentials. I'm here to mansplain' some wisdom on you - because I'm gettin' mad.

Here's what we want:

git clone ssh://git@example.com/org/project.git

Cloning into 'project'...

Now let's talk about what we don't want.

We don't want this:

git clone ssh://git@github.com/example/project.git

git@github.com: Permission denied (publickey).

fatal: Could not read from remote repository.

And we don't want this either:

git clone https://github.com/example/project.git

Username for 'https://github.com':

Password for 'https://github.com':

We want to be able to clone private repos without a password or passphrase prompt.

Why do we want this?

I dunno.

For me, I much prefer to simply use git, which every package manager in the Universe supports, than to have hacky one-off solutions or use paid private package registries where a simple repo url in a package.json, requirements.txt, go.mod, or Gemfile will do.

TL;DR & Cheatsheet

You know what else we all want? A magic one-liner that fixes our problems.

Fortunately, in this case, I have one for you:

GIT_TOKEN=xxxxx
git config --global url."https://ssh:$GIT_TOKEN@github.com/".insteadOf "ssh://git@github.com/"

Technically that's two lines, but one of them would be in your environment variable configs, so it doesn't count. Anyway, season to taste and that alone will work for reasonably well for almost any flavor of deployment that you need.

If you're just a cheat looking for a quick fix, you might like

• The Git Credentials Cheatsheet.

However, if you want detail, then continue reading below. The insteadOf method shown above specifically is in the .gitconfig section up ahead.

0. Mental Bookmark & ToC

This is "The Vanilla DevOps Guide to Git Credentials: Ultimate Collection".

That's too long of course, so I couldn't fit it all in the title, but that's what this really is aiming to be.

• "Vanilla DevOps Guide" because the focus is on the simple, easy, old-fashioned, boring way.
• "Git Credentials" because that's what authenticing with git platforms like github is called.
• "Ultimate Collection" because we're covering all known methods, and the bits that bite.

The Internet's a big place, but good keywords still work half of the time. I'm putting this together for posterity (my future self, for example) and I wanted to make sure it's easy to remember and search for.

Specifically, we're going to cover all of this:

• Git+SSH: Public Keys
  • Why to use them for Local Development
  • Why to NOT use them for Deployments
• Git Acccess Tokens
  • GIT_ASKPASS
    • SSH_ASKPASS
  • .gitconfig (using insteadOf)
  • git-credentials
  • .netrc
• Passwords

Git+SSH: Public Keys

SSH Authentication is the primary way that devs authenticate with git for normal, everyday, local git stuff.

When you see stuff like this:

git clone ssh://git@github.com/example/project.git

git clone git@github.com:example/project.git

That's SSH authentication.

SSH for Local Development

When it comes to your local computer, you should be using SSH Public Key Authentication.

No Discussion. End of Story. The End.

Got it? Good.

Essentially you run this, pick a passphrase (anything really), and copy the output to Account > Settings > SSH and GPG keys in Github (or wherever):

# Show or generate your public key (accept the defaults)
[ ! -f "$HOME/.ssh/id_rsa.pub" ] && ssh-keygen && echo "# copy below"
cat ~/id_rsa.pub

You don't need to bother remembering your passphrase, because you can save it, encrypted, in your OS keychain (which gets unlocked when you login):

# So you can forget your passphrase
ssh-add ~/id_rsa

The nice thing about SSH keys is... everything. I just makes your life 100x easier.

• SSH Keys are very secure (maths, randomness, bit entropy, prime numbers, and all that)
• SSH Keys are intended to be long-lived (years)
• SSH Keys are per-device (and supposed to be generated on-device)
• SSH Keys are protected by passphrases (which, as stated, you don't have to remember!)

I won't waste any more time on that, other than to say that if you want long explanations with pictures and stuff you can check out Github's guides. They're not bad and, since most git platforms (Gitea, Gitlab, Bitbucket, etc) are pretty much the same, this stuff applies to everything:

• Connecting to Github with SSH
  • Checking for existing SSH keys
  • Adding a new SSH key to your GitHub account
  • Generating a new SSH key and adding it to the ssh-agent

SSH for Deployment Pipelines

For certain On Premise style deployments, using SSH Keys can make sense.

But for anything Docker, serverless, "cloud scale"... forget it - the exact opposite is true: You shouldn't use SSH keys for ephemeral cloud deployments.

I repeat:

You shouldn't use SSH keys for ephemeral cloud deployments.

Now not everyone agrees about this, but they're wrong, and it's easy to see that they're wrong: It's complicated. And anything complicated is inherently insecure.

This is a law kinda like Godwin's law, but to the effect of

As security practices get more stupid and complicated, people just do more stupid things to subvert them, or simply get it wrong.

Using multiline environment variable config strings, or committing the keys with passphrase protection, or whatever else... all fine if you have to do them (when the platform forces you to, or has premission features not otherwise available), but otherwise you're better off just to do things the easy way instead.

Whenever you have to over-engineer a solution, you're contributing to the problem.

Bottom line: don't do it. Even for hacky ways, there are less hacky ways. That's why the rest of this article exists!

Exception: When platform you're working with forces you to use them, don't try to hack around it, just go with the flow. And probably use a passphrase.

Access Tokens

Access Tokens are the alternative SSH Keys. They're sent over HTTPS using "Basic" Authorization.

IMPORTANT: It's hard to test if this is working or not due to credential caching (you'll get lots of false positives). See the note at the bottom of the git-credentials section for more info.

git clone https://token:xxxxxxxxxxxxxxxx@github.com/example/project.git

You can store them in Environment variables or password files on disk. Ideally you want them to be available to your build pipeline, but not your deploy pipleline but, again, cloud stuff is all ephemeral and sandboxed anyway.

Depending on the platform these are likely to be called something to the effect of "Access Token", "Server Token" or "API Key".

It is NOT "Private Key", "OAuth2 Secret", or "Certificate".

Some of the other terms (such as "deployment key") are more ambiguous as to what they refer to - you may have to read the docs.

Anyway, you have a few different options on how to present such a token:

• GIT_ASKPASS
• .gitconfig
• .git-credentials
• .netrc

GIT_ASKPASS

Never leave your ENVs

I'll mention GIT_ASKPASS and SSH_ASKPASS first because they have the benefit of being able to use an API Token in an Environment config without actually writing any secrets to disk.

It works like this:

Wherever git clones an authenticated https repository and gets a 401 Needs Authentication, it first checks the GIT_ASKPASS environment variable to see if you've specified a program (a script in our case) that can present the API Token (or password in the case of Desktop applications like VSCode, Keybase, etc).

If the ENV exists, and the Token in returns works, it doesn't bother you further. Otherwise you get the password prompt

The Access Token could be stored directly in that script, or it could be stored in a separate file read by the script, or in a database. Wherever.

My example here is as if it's 2020 and we're all bleeding environment variables everywhere.

You've got your ENVs

.env:

GIT_TOKEN=xxxxxxxxxxxxxxxx
GIT_ASKPASS=/app/.git-askpass

(you can also git config --global core.askPass /app/.git-askpass)

And a simple one-line script that echoes the Token:

.git-askpass:

echo $GIT_TOKEN

Bonus: The program will receive a password prompt on stdin, which could be useful to parse so as to provide different tokens for different hosts.

Password for 'proto://host.tld':

It also receives a few special environment variablies:

GIT_DIR=/Users/me/project/.git
GIT_EXEC_PATH=/usr/local/Cellar/git/2.19.0_1/libexec/git-core
GIT_PREFIX=

And, of course, the script has to be executable:

chmod 0700 ~/.git-askpass

You can try that out on your own. It'll work.

If you're Dockerizing, you'll probably do something like this:

Dockerfile:

FROM alpine:latest as build

WORKDIR /app
RUN echo 'echo $GIT_TOKEN' > .git-askpass && \
  chmod +x .git-askpass
RUN git clone https://api@github.com/example/project.git

Just know that, in the age of Docker / CircleCI / Drone / Heroku / Akkeris / whatever build & deploy pipeline serverles "cloud scale", all of your ENVs are still written to disk and persissted on the very service that's hosting the images and management console (duh), and it's not like anyone (or even you) can reasonably remote into these containers anyway, so most of the talk about "oh no! don't write secrets to disk" is just FUD leftover from the 1990s in relation to magnetic drives of easy-to-steal / break into physical devices.

And thus we move on...

SSH_ASKPASS

This is almost exactly the same as the above, but for ssh urls and passphrase-protected ssh keys.

For deployments an SSH "Passphrase" should actually be a simple 128-bit crypto-random hex or base64 string, not a phrase of words.

It's useful when you're in a situation where you're authenticating with a shared (passphrase-protected) private ssh key. Although frowned upon, it's actually perfectly secure to commit to a repo or even make completely public - as long as it's protected by a crypto-random passphrase.

It's every whit as secure (in entropy) to use as an Access Token, just more cumbersome and therefore not a good idea when an alternative is readily available (which it usually is).

SSH_PASSPHRASE=xxxxxxxxxxxxxxxx
SSH_ASKPASS=$HOME/.ssh-askpass

And a simple one-line script that echoes the Token:

echo 'echo $SSH_PASSPHRASE' > ~/.ssh-askpass
chmod +x .ssh-askpass

.gitconfig (easiest)

I find the .gitconfig "insteadOf" directive to be the easiest approach, and it has some marked advantages over GIT_ASKPASS:

• Manage multiple credentials
• Interchange ssh, git, and https urls
• Granular path pattern matching (per org, project, platform, etc)
• Doesn't require manual file writes

If you wanted to make any version of a Github URL use your access token, you can do that like this:

GIT_TOKEN=xxxxxxxxxxxxxxxx

git config --global url."https://token:$GIT_TOKEN@github.com/".insteadOf "ssh://git@github.com/"
git config --global url."https://token:$GIT_TOKEN@github.com/".insteadOf "git@github.com:"
git config --global url."https://token:$GIT_TOKEN@github.com/".insteadOf "https://github.com/"
git config --global url."https://token:$GIT_TOKEN@github.com/".insteadOf "https://api:github.com/"

And it'll produce this file:

.gitconfig:

[url "https://api:xxxxxxxx@github.com/"]
  insteadOf = https://github.com/
[url "https://api:xxxxxxxx@github.com/"]
  insteadOf = https://api@github.com/
[url "https://api:xxxxxxxx@github.com/"]
  insteadOf = ssh://git@github.com/
[url "https://api:xxxxxxxx@github.com/"]
  insteadOf = git@github.com:

If you needed different credentials for different organizations or projects, you could also handle that quite easily:

git config --global url."https://token:$GIT_TOKEN@github.com/example/".insteadOf "ssh://git@github.com/example/"
git config --global url."https://token:$GIT_TOKEN@github.com/other/project".insteadOf "ssh://git@github.com/other/project"

As you can see, the ways these work is a little bit backwards. You can tweak it by hand to allow duplicate keys or, since the token part is all that matters (not the username), you can set the username part (api:) to any arbitrary string you wish.

git-credentials

This is pretty similar to "insteadOf" approach above and you can mix and match them together.

IMPORTANT: It's hard to test if this is working or not (lots of false positives), see the note at the bottom of this section for more info

Enable the plaintext credential store:

git config --global credential.helper store

~/.git-config:

[credential]
    helper = store

Enter credentials for it to save:

echo "protocol=https
host=github.com
path=example/
username=token
password=$GIT_TOKEN" | git credential-store --file $HOME/.git-credentials store

~/.git-credentials:

https://token:xxxxxxx@github.com/myorganization/

You can test which credentials git will respond with (after all "insteadOf" replacements) similarly:

echo "protocol=https
host=github.com
path=example/
username=token" | git credential fill

Bonus: You can write your own git-credential plugin / tool: https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage

Testing that git-credentials are working can be frustrating: The default behavior is to cache working results, which makes it hard to test what's working on your machine vs what will work in deployment. The best option is probably to disable all --local, --global, and --system credential helpers until you can be sure that the only ones at play are the ones you've chosen.

# See what credential rules are in place
git config --show-origin --get credential.helper

# Disable caching on MacOS (reverse to re-enable)
sudo mv /Applications/Xcode.app/Contents/Developer/usr/share/git-core/gitconfig /Applications/Xcode.app/Contents/Developer/usr/share/git-core/gitconfig.bak

# Disable user credential caching
rsync -av ~/.gitconfig ~/.gitconfig.bak
git config --global --unset credential.helper

# Disable local credential caching for a project
rsync -av .git/config .git/config.bak
git config --local --unset credential.helper

.netrc

Lastly, there's good, old, 1990s .netrc, which is a generic file used by curl, git, ftp, as well as, I dunno, probably a good half of all the popular CLI networking tools from back then.

It's a well-known and easy to use option, and has slightly more granularity than our old friend GIT_ASKPASS, but doesn't respect paths.

~/.netrc:

machine github.com
login token
password xxxxxxxx

Passwords

Passwords are for ignoramuses. Don't be an ignoramus.

But in all seriousness... that's how it is. Don't use passwords. Use either keys or tokens.

Hipster Security Considered Harmful

Don't fall victim to trendy hipster security.

Focusing anywhere other than your weakest link is an illusion. Your weakest link isn't in the bit-entropy of SSH, or the security of cloud systems, or in Github's API Tokens - it's with your people and your processes.

Those things are fun to goof off with and theorize about, but tooling with them makes things more complicated - and when it's more complicated, it's less secure.

Make it easy for your people to do their job and you'll be less likely to lose your job.

Always remember:

You've got to make it easier to do the right thing than the wrong thing, or even nothing at all.

Also:

Don't be stupid. Keep it Stupid Simple - because that's secure.

By AJ ONeal


Did I make your day?
Buy me a coffee  

(you can learn about the bigger picture I'm working towards on my patreon page )

---

---