Use the netrc credential helper for your git https passwords

I have been recently hacking some Golang and fiddling with its package management. In all honesty, I quite like the idea of using GitHub as an artifact storage for its immediateness and simplicity and it comes as no surprise that other languages are offering the same capability 1.

This does come with some complexity though and unexpected problems might creep in. I put together this blog post to show how to solve a particular problem I encountered around GitHub https access, as opposed to ssh access, that the go tool requires for fetching dependencies.

The problem

In a nutshell, whenever a go package relies on a file residing in a private GitHub repository and https is the only option you have for accessing it, commands like go get can spit the quite cryptic:

fatal: could not read Username for 'https://github.com': terminal prompts disabled

This is a well known problem and there is even an FAQ entry in the official Go documentation with two solutions.

In this post we are going to expand on the first solution, that is to store username and password in your $HOME/.netrc.

Store credentials on your hard drive

Not a security expert here, but when I first read the FAQ above it seemed quite naughty to save credentials in plain text on my machine. The obvious solution is encryption.

However, if private keys and encrypted files live on the same hard drive the protection that encryption gives you is limited. If an attacker steals your laptop she will be able to recover your secrets. Using a (strong) password for your keys is more secure because it usually means that your private key is encrypted against it at rest.

I personally found the pwgen command to be a good companion here:

The pwgen program generates passwords which are designed to be easily memorized by humans, while being as secure as possible. Human-memorable passwords are never going to be as secure as completely random passwords.

An even more secure setup would be to avoid the physical co-location of the above-mentioned files. This is out of scope here but I will link to a series of neat setups at the end of the post.

Encrypt .netrc with GnuPG

Let's start by creating and populating the .netrc file.

mv ~/.netrc ~/.netrc.old          # just in case
cat > ~/.netrc
machine github.com login USERNAME password APIKEY protocol https
<CTRL+D>

Replace the last line with your actual GitHub credentials 2.

Now, there are numerous ways to encrypt files in the world but because of its quasi-out-of-the-box support we are going to explore the GnuPG option.

Setting it up from scratch is out of scope but the GnuPG Getting Started is quite easy to follow...don't forget the password!

Assuming your setup works and you used user@email.com for the email address, you can now encrypt to yourself:

gpg --encrypt --recipient user@email.com ~/.netrc

This will produce the .netrc.gpg file in the $HOME dir.

If you want to verify it worked you can decrypt and pipe to diff like so:

 gpg --decrypt --recipient user@email.com ~/.netrc.gpg | diff -s ~/.netrc -
 ...
 Files /home/user/.netrc and - are identical

Git Credential Helpers

Fortunately for us, the git team has figured out the credential problem for us: there is full support for both storing and caching through Credential helpers.

We are going to use a custom credential helper called git-credential-netrc 3. It is maintained directly in the git repository as part of its Contributed Software.

We will build the binary first and then move it to our favorite bin folder:

git clone https://github.com/git/git.git
cd git
git checkout 6579d93a9789448c54203939650ca24409635f80
cd contrib/credential/netrc
make
cp -v git-credential-netrc `$HOME/.local/bin` # on my $PATH

You might need to use sudo for the last command. Note that we check out a specific commit there: things are always in-flux and hopefully today's working snippet will also work tomorrow.

At this point the only missing piece is to tell git to use the above binary as credential helper:

git config --global credential.helper netrc

Note that git already prepends git-credential- to its helpers, that is why you don't see it above. I scratched my head for a little while there.

Another error I found in the wild that did not happen here is "Inappropriate ioctl for device".

For solving that make sure you do the following:

cat export GPG_TTY=$(tty) >> ~/.bashrc

That's all folks

The next time you pull from a private https://github.com repository, the credential helper will kick in, decrypting .netrc.gpg so that git can make use of the username and password.

As promised, here are very nice ideas around GnuPG private key air-gapping:

I hope this post saved you some Googling time.

Thank you for reading!


  1. I still slightly prefer Clojure team's choice to optionally source Git dependencies, along with Maven.

  2. Achievement unlocked! Show the cat > file trick in a blog post.

  3. Other helpers exist. The git-credential-osxkeychain listed here might be useful to macOS users.