The Ultimate Guide to Git CredentialsPublished 2019-7-24
There's more than one way to skin a cat, and there's also more than one way to do a lot of other things - like store and cache your git credentials. I'm here to mansplain' some wisdom on you - because I'm gettin' mad.
Here's what we want:
git clone ssh://email@example.com/org/project.git
Cloning into 'project'...
Now let's talk about what we don't want.
We don't want this:
git clone ssh://firstname.lastname@example.org/example/project.git
email@example.com: Permission denied (publickey).
fatal: Could not read from remote repository.
And we don't want this either:
git clone https://github.com/example/project.git
Username for 'https://github.com':
Password for 'https://github.com':
We want to be able to clone private repos without a password or passphrase prompt.
Why do we want this?
For me, I much prefer to simply use
which every package manager in the Universe supports,
than to have hacky one-off solutions or use paid private package registries
where a simple repo url in a
Gemfile will do.
TL;DR & Cheatsheet
You know what else we all want? A magic one-liner that fixes our problems.
Fortunately, in this case, I have one for you:
GIT_TOKEN=xxxxx git config --global url."https://ssh:$GIT_TOKEN@github.com/".insteadOf "ssh://firstname.lastname@example.org/"
Technically that's two lines, but one of them would be in your environment variable configs, so it doesn't count. Anyway, season to taste and that alone will work for reasonably well for almost any flavor of deployment that you need.
If you're just a cheat looking for a quick fix, you might like
However, if you want detail, then continue reading below. The insteadOf method shown above specifically is in the .gitconfig section up ahead.
0. Mental Bookmark & ToC
This is "The Vanilla DevOps Guide to Git Credentials: Ultimate Collection".
That's too long of course, so I couldn't fit it all in the title, but that's what this really is aiming to be.
- "Vanilla DevOps Guide" because the focus is on the simple, easy, old-fashioned, boring way.
- "Git Credentials" because that's what authenticing with git platforms like github is called.
- "Ultimate Collection" because we're covering all known methods, and the bits that bite.
The Internet's a big place, but good keywords still work half of the time. I'm putting this together for posterity (my future self, for example) and I wanted to make sure it's easy to remember and search for.
Specifically, we're going to cover all of this:
- Git+SSH: Public Keys
- Why to use them for Local Development
- Why to NOT use them for Deployments
- Git Acccess Tokens
Git+SSH: Public Keys
SSH Authentication is the primary way that devs authenticate with git for normal, everyday, local git stuff.
When you see stuff like this:
git clone ssh://email@example.com/example/project.git
git clone firstname.lastname@example.org:example/project.git
That's SSH authentication.
SSH for Local Development
When it comes to your local computer, you should be using SSH Public Key Authentication.
No Discussion. End of Story. The End.
Got it? Good.
Essentially you run this, pick a passphrase (anything really),
and copy the output to
Account > Settings > SSH and GPG keys in Github (or wherever):
# Show or generate your public key (accept the defaults) [ ! -f "$HOME/.ssh/id_rsa.pub" ] && ssh-keygen && echo "# copy below" cat ~/id_rsa.pub
You don't need to bother remembering your passphrase, because you can save it, encrypted, in your OS keychain (which gets unlocked when you login):
# So you can forget your passphrase ssh-add ~/id_rsa
The nice thing about SSH keys is... everything. I just makes your life 100x easier.
- SSH Keys are very secure (maths, randomness, bit entropy, prime numbers, and all that)
- SSH Keys are intended to be long-lived (years)
- SSH Keys are per-device (and supposed to be generated on-device)
- SSH Keys are protected by passphrases (which, as stated, you don't have to remember!)
I won't waste any more time on that, other than to say that if you want long explanations with pictures and stuff you can check out Github's guides. They're not bad and, since most git platforms (Gitea, Gitlab, Bitbucket, etc) are pretty much the same, this stuff applies to everything:
- Connecting to Github with SSH
SSH for Deployment Pipelines
For certain On Premise style deployments, using SSH Keys can make sense.
But for anything Docker, serverless, "cloud scale"... forget it - the exact opposite is true: You shouldn't use SSH keys for ephemeral cloud deployments.
You shouldn't use SSH keys for ephemeral cloud deployments.
Now not everyone agrees about this, but they're wrong, and it's easy to see that they're wrong: It's complicated. And anything complicated is inherently insecure.
This is a law kinda like Godwin's law, but to the effect of
As security practices get more stupid and complicated, people just do more stupid things to subvert them, or simply get it wrong.
Using multiline environment variable config strings, or committing the keys with passphrase protection, or whatever else... all fine if you have to do them (when the platform forces you to, or has premission features not otherwise available), but otherwise you're better off just to do things the easy way instead.
Whenever you have to over-engineer a solution, you're contributing to the problem.
Bottom line: don't do it. Even for hacky ways, there are less hacky ways. That's why the rest of this article exists!
Exception: When platform you're working with forces you to use them, don't try to hack around it, just go with the flow. And probably use a passphrase.
Access Tokens are the alternative SSH Keys. They're sent over HTTPS using "Basic" Authorization.
IMPORTANT: It's hard to test if this is working or not due to credential caching
(you'll get lots of false positives). See the note at the bottom of the
git-credentials section for more info.
git clone https://token:email@example.com/example/project.git
You can store them in Environment variables or password files on disk. Ideally you want them to be available to your build pipeline, but not your deploy pipleline but, again, cloud stuff is all ephemeral and sandboxed anyway.
Depending on the platform these are likely to be called something to the effect of "Access Token", "Server Token" or "API Key".
It is NOT "Private Key", "OAuth2 Secret", or "Certificate".
Some of the other terms (such as "deployment key") are more ambiguous as to what they refer to - you may have to read the docs.
Anyway, you have a few different options on how to present such a token:
Never leave your ENVs
SSH_ASKPASS first because
they have the benefit of being able to use an API Token in
an Environment config without actually writing any secrets to disk.
It works like this:
git clones an authenticated https repository and gets a 401 Needs Authentication,
it first checks the
GIT_ASKPASS environment variable to see if you've specified a program
(a script in our case) that can present the API Token (or password in the case of Desktop applications
like VSCode, Keybase, etc).
If the ENV exists, and the Token in returns works, it doesn't bother you further. Otherwise you get the password prompt
The Access Token could be stored directly in that script, or it could be stored in a separate file read by the script, or in a database. Wherever.
My example here is as if it's 2020 and we're all bleeding environment variables everywhere.
You've got your ENVs
(you can also
git config --global core.askPass /app/.git-askpass)
And a simple one-line script that echoes the Token:
Bonus: The program will receive a password prompt on stdin, which could be useful to parse so as to provide different tokens for different hosts.
Password for 'proto://host.tld':
It also receives a few special environment variablies:
GIT_DIR=/Users/me/project/.git GIT_EXEC_PATH=/usr/local/Cellar/git/2.19.0_1/libexec/git-core GIT_PREFIX=
And, of course, the script has to be executable:
chmod 0700 ~/.git-askpass
You can try that out on your own. It'll work.
If you're Dockerizing, you'll probably do something like this:
FROM alpine:latest as build WORKDIR /app RUN echo 'echo $GIT_TOKEN' > .git-askpass && \ chmod +x .git-askpass RUN git clone https://firstname.lastname@example.org/example/project.git
Just know that, in the age of Docker / CircleCI / Drone / Heroku / Akkeris / whatever build & deploy pipeline serverles "cloud scale", all of your ENVs are still written to disk and persissted on the very service that's hosting the images and management console (duh), and it's not like anyone (or even you) can reasonably remote into these containers anyway, so most of the talk about "oh no! don't write secrets to disk" is just FUD leftover from the 1990s in relation to magnetic drives of easy-to-steal / break into physical devices.
And thus we move on...
This is almost exactly the same as the above, but for ssh urls and passphrase-protected ssh keys.
For deployments an SSH "Passphrase" should actually be a simple 128-bit crypto-random hex or base64 string, not a phrase of words.
It's useful when you're in a situation where you're authenticating with a shared (passphrase-protected) private ssh key. Although frowned upon, it's actually perfectly secure to commit to a repo or even make completely public - as long as it's protected by a crypto-random passphrase.
It's every whit as secure (in entropy) to use as an Access Token, just more cumbersome and therefore not a good idea when an alternative is readily available (which it usually is).
And a simple one-line script that echoes the Token:
echo 'echo $SSH_PASSPHRASE' > ~/.ssh-askpass chmod +x .ssh-askpass
I find the
.gitconfig "insteadOf" directive to be the easiest approach,
and it has some marked advantages over
- Manage multiple credentials
- Interchange ssh, git, and https urls
- Granular path pattern matching (per org, project, platform, etc)
- Doesn't require manual file writes
If you wanted to make any version of a Github URL use your access token, you can do that like this:
git config --global url."https://token:$GIT_TOKEN@github.com/".insteadOf "ssh://email@example.com/" git config --global url."https://token:$GIT_TOKEN@github.com/".insteadOf "firstname.lastname@example.org:" git config --global url."https://token:$GIT_TOKEN@github.com/".insteadOf "https://github.com/" git config --global url."https://token:$GIT_TOKEN@github.com/".insteadOf "https://api:github.com/"
And it'll produce this file:
[url "https://api:email@example.com/"] insteadOf = https://github.com/ [url "https://api:firstname.lastname@example.org/"] insteadOf = https://email@example.com/ [url "https://api:firstname.lastname@example.org/"] insteadOf = ssh://email@example.com/ [url "https://api:firstname.lastname@example.org/"] insteadOf = email@example.com:
If you needed different credentials for different organizations or projects, you could also handle that quite easily:
git config --global url."https://token:$GIT_TOKEN@github.com/example/".insteadOf "ssh://firstname.lastname@example.org/example/" git config --global url."https://token:$GIT_TOKEN@github.com/other/project".insteadOf "ssh://email@example.com/other/project"
As you can see, the ways these work is a little bit backwards.
You can tweak it by hand to allow duplicate keys or,
since the token part is all that matters (not the username), you can set the username part (
to any arbitrary string you wish.
This is pretty similar to "insteadOf" approach above and you can mix and match them together.
IMPORTANT: It's hard to test if this is working or not (lots of false positives), see the note at the bottom of this section for more info
Enable the plaintext credential store:
git config --global credential.helper store
[credential] helper = store
Enter credentials for it to save:
echo "protocol=https host=github.com path=example/ username=token password=$GIT_TOKEN" | git credential-store --file $HOME/.git-credentials store
You can test which credentials git will respond with (after all "insteadOf" replacements) similarly:
echo "protocol=https host=github.com path=example/ username=token" | git credential fill
Bonus: You can write your own git-credential plugin / tool: https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage
Testing that git-credentials are working can be frustrating: The default behavior is to cache working results,
which makes it hard to test what's working on your machine vs what will work in deployment.
The best option is probably to disable all
--system credential helpers
until you can be sure that the only ones at play are the ones you've chosen.
# See what credential rules are in place git config --show-origin --get credential.helper # Disable caching on MacOS (reverse to re-enable) sudo mv /Applications/Xcode.app/Contents/Developer/usr/share/git-core/gitconfig /Applications/Xcode.app/Contents/Developer/usr/share/git-core/gitconfig.bak # Disable user credential caching rsync -av ~/.gitconfig ~/.gitconfig.bak git config --global --unset credential.helper # Disable local credential caching for a project rsync -av .git/config .git/config.bak git config --local --unset credential.helper
Lastly, there's good, old, 1990s
.netrc, which is a generic file used by
as well as, I dunno, probably a good half of all the popular CLI networking tools from back then.
It's a well-known and easy to use option, and has slightly more granularity than
our old friend
GIT_ASKPASS, but doesn't respect paths.
machine github.com login token password xxxxxxxx
Passwords are for ignoramuses. Don't be an ignoramus.
But in all seriousness... that's how it is. Don't use passwords. Use either keys or tokens.
Hipster Security Considered Harmful
Don't fall victim to trendy hipster security.
Focusing anywhere other than your weakest link is an illusion. Your weakest link isn't in the bit-entropy of SSH, or the security of cloud systems, or in Github's API Tokens - it's with your people and your processes.
Those things are fun to goof off with and theorize about, but tooling with them makes things more complicated - and when it's more complicated, it's less secure.
Make it easy for your people to do their job and you'll be less likely to lose your job.
You've got to make it easier to do the right thing than the wrong thing, or even nothing at all.
Don't be stupid. Keep it Stupid Simple - because that's secure.
By AJ ONeal
Did I make your day?
Buy me a coffee