Handling authentication with Composer and private Github repositories

Estimated Reading Time: 11min

If you’re working on anything with a stack managed by Composer you’ll probably sooner or later hit the point where you’ll need to add some private packages to your project, often from Github.
In this post I’ll walk you through multiple approaches to handle this situation, from simple to complex, depending on your use case.

One Github account

The most basic approach is to add a private repository:

{
    "require": {
        "vendor/my-private-repo": "dev-master"
    },
    "repositories": [
        {
            "type": "vcs",
            "url": "git@github.com:vendor/my-private-repo"
        }
    ]
}

This has only one requirement: The machine running composer needs to have SSH access to the Git repository. Satisfying this requirement is, under certain circumstances, not that difficult.

For Github this means generating a new SSH key (unless you already have one) and then adding it to your GitHub account.

This also works if you add multiple repositories from the same Github account.

Read-only access with deploy keys

The above approach uses a Github account level SSH key. This might be fine for your local machine, but since those keys have full read and write access to all your Github repositories it probably isn’t advisable for build- or web servers. Since those never need to change the code, just access it, giving them full access adds an unnecessary security risk. What we’d like to have instead for those is read-only access.

Currently Github only offers one way to create read-only keys: deploy keys

The process for adding a read-only deploy key is similiar to the one for an account SSH key.

But since Github doesn’t allow the same key to be used for multiple repositories you’ll end up with one key per repository. This will leave us with the same issue we’ll have with multiple Github accounts.

Multiple Github accounts

If you want to add another private repository from another Github account you might think that just generating another SSH key and adding that to the other Github account should work. But as you’ll soon notice you’ll run into a problem:

Since you always connect to Github as the user git, SSH will always connect using the same SSH key. The connection will work, but this key will only ever give you access to one account (or repository if you’re using deploy keys).

SSH Aliases

If you do a quick search on the web the approach that people will recommend most often is creating multiple aliases in your ~/.ssh/config

Host repo1
    HostName github.com
    User git
    IdentityFile ~/.ssh/repo1
    IdentitiesOnly yes

Host repo2
    HostName github.com
    User git
    IdentityFile ~/.ssh/repo2
    IdentitiesOnly yes

and then using those aliases in your composer.json:

{
    "repositories": [
        {
            "type": "vcs",
            "url": "repo1:vendor/my-private-repo.git"
        },
        {
            "type": "vcs",
            "url": "repo2:vendor2/my-private-repo-2.git"
        }
    ]
}

This works but has major downsides:

  1. The composer.json doesn’t contain the real github address any more (repo1:vendor/my-private-repo.git instead of git@github.com:vendor/my-private-repo) and hence (apart from being confusing) only works when everyone working with the project has defined the same alias in the ~/.ssh/config file.
  2. Therefore you’ll need to manually maintain and keep in sync the ~/.ssh/config for all machines/developers.
  3. If you ever need to change the aliases (think having colliding aliases over multiple projects when using something generic like the above repo1) you’ll need to update the composer.json and break access for any historic references to the old alias.

After some further research I came across multiple ideas on how you might work around those issues. I’m adding this mostly for myself to document my research process. You can probably skip over this list since none of these approaches really turned out to be a good solution.

  • Manually configured SSH keys in your composer.json would kind of circumvent issue 1. I never bothered getting this to work though since you need the SSH2 PECL which is not commonly installed. Even if this worked you’d still be manually maintaining SSH key configuration, just in your composer.json instead of the ~/.ssh/config.
  • Adding a wildcard CNAME DNS record to e.g. your companies domain that points to github, something like *.github.example.com. CNAME github.com. This way you’d still need to configure SSH keys for each repository in your ~/.ssh/config but at least you could better enforce canonical and consistent aliases since those are real globally accessible URLs backed by public DNS records. Still all it does is simplify issue 2 a bit.
  • Using the url.<base>.insteadOf setting in git-config you could automatically, internally rewrite Git URLs to something that matches your locally configured alias. Since this works on Git level you can keep the real Github URLs for Composer basically solving issue 1 and 3. But it adds another configuration file you need to manually maintain locally for each repository.

All of these solutions only solve part of the problems with using SSH aliases and none does it in a simple and elegant way. The third one looked the most promising one and combined with some helper script that automatically alters entries from ~/.ssh/config and git-config might even lead to a working solution. But it is a quite complex setup and hence probably not ideal for everyday use.

But still the last solution put me on track to see what can be done with Git since up until here I was mostly looking for solutions in the realm of Composer and SSH…

Wrapping SSH with GIT_SSH_COMMAND

Up until here I was trying to find a way to configure Composer and/or SSH to use different keys for different repositories. But since Git is a great and flexible tool I then found a solution that can make Git itself choose different SSH keys: the GIT_SSH_COMMAND

The way it works is that you can set an arbitrary executable that gets called instead of the plain ssh whenever Git needs to make a SSH connection. This script could then call SSH setting a SSH key using the -i option of ssh just passing all parameters to SSH:

GIT_SSH_COMMAND="ssh -i ~/.ssh/vendor_1_ssh_key -o IdentitiesOnly=yes" /usr/bin/git $@

How such a script determines which key to use is quite flexible and totally up to you.

If you just have different Github accounts for different projects you could for example make the decision based on the working folder.

In the next section I’ll explain another approach that automates the decision which key to use even further.

Repository-name based key selection

As already mentioned earlier when using deploy keys we not only need to maintain one key per Github account but one per repository.

Switching the key based on the working folder as mentioned above forces us to manually maintain a long list of folder path -> key mappings that probably differ from machine to machine and hence are hard to maintain and keep in sync with a team. So we should probably look for a way to make this easier.

After some searching and thinking what I found was a very smart approach that harnesses the fact that Git passes the path of the repository to the script which we can use to automatically search for a key matching the name and use that.

This leaves us with a solution where for adding a private repository we

… add the github repository to our composer.json without any alias-magic …

{
    "repositories": [
        {
            "type": "vcs",
            "url": "git@github.com:vendor/my-private-repo"
        }
    ]
}

add one deploy key per Github repository named based on the repository name …

~/.ssh/git-keys/vendor-my-private-repo

… and globally set a wrapper that automatically picks the right key based on the repository name.

git config --global core.sshCommand /path/to/your/wrapper.sh

One could probably automate this even further with a bash script up until creating the deploy keys via the Github API.

Project level package repository

All of this already got us quite far, but we still need to create plenty of deploy keys. At least one for each repository. Even more so if we have multiple machines that need read only access. Also we still need to configure each Github repository for each package in each projects composer.json.

So if we don’t reuse deploy keys across clients and machines, if all in all we have 15 client projects, each with 3 private packages for that project that means at least 45 deploy keys to juggle manually for each machine.

If we’re making wishes for a moment, I’d love a solution where we just add one composer repository and corresponding pair of credentials per project and have all private packages just automatically work.

Private Packagist

Apparently I am not the only one with this wish, so that is why the creators of Composer have built Private Packagist as a service where you can host private packages with granular access control. This sounds great (if not only for being a way to financially support the development of Composer) and apparently checks all those marks (and many more!). You should definitely have a look at it and see if it fits your needs!

For me personally though it has two one issue:

  1. The pricing is a bit heavy.
  2. It’s a SaaS solution unless you pay even more.

For a one-person business like mine that has many small clients I’d still need the agency plan. For, let’s say 15 clients, this would cost me around €2500 per year. So if I need this long term I can put quite some time into an alternative solution.

After a conversation on Twitter with Nils Adermann from Private Packagist I’ve found out that for freelancers that only need one human user there is a way cheaper (€219/year) option that is only available after you contact them.
Also there is an alternative approach that doesn’t necessarily need subrepositories to separate packages between clients at all.
I’m currently evaluating this myself, it’s looking very promising and I’ll probably go into detail on this in another post but wanted to rectify the information in this post a.s.a.p..

A home-grown solution on top of Satis

Luckily as an alternative to the public packagist.org and the aforementioned Private Packagist there also is the simpler, self-hosted, static Composer repository generator Satis.

But in comparison to Private Packagist it has multiple major limitations for which we’d need to build solutions ourselves:

  • no concept of access control whatsoever
  • no automatic rebuilding of the Composer repository…
  • … especially not on webhooks triggered by e.g. git push

How Satis works would go beyond the scope of this post but the documentation on how to build your Composer repository with Satis should get you pretty far.

The way I would set this up is to create one repository per client or project that contains everything needed for it.

Satis is built on top of Composer, so to access private repositories everything in this post applies. The big advantage is that you only need to configure the server that runs Satis to have access with deploy keys, not each and every build-/web server or developer machine.

Once you’ve got your Satis-built Composer repository that contains all packages for the project up all you need to do is add it to your project:

{
  "repositories": [
    {
      "type": "composer",
      "url": "https://packages.example.org/project-name"
    }
  ]
}

Again, be aware that Satis itself doesn’t have any access control, so anyone can see and use the packages if you don’t add authentication on top.

If you’re web server is Apache you should be able do this quite easily though by adding Basic Auth to the Satis folder. You’ll find plenty of tutorials for this online.

When it comes to automatic rebuilding you could set up a cron job that runs in regular intervals. (The linked tutorial is targeted at users of the web hosting provider uberspace but the general approach is the same everywhere)

What would also be nice to have is a webhook endpoint that gets called whenever any code gets pushed on your private packages Github repository and automatically rebuilds the package repository.

The first time Composer/Satis connects to your package repository it will ask for your Basic Auth credentials which can then be stored in your auth.json file so Composer doesn’t need to ask for them again.

TL;DR

So to summarize the pieces needed to build a very basic, self-hosted alternative for the core features of Private Packagist you’d need to combine those pieces:

  • a Git wrapper that automatically does pick Github deploy keys based on the repository name (see above)
  • one Satis built Composer repository per client/project
  • access control with e.g. Apache Basic Auth with possibly multiple users per repository
  • a cron job for regular rebuilding
  • a webhook endpoint for on-demand rebuilds of specific repositories/packages

For myself I’m currently in the process of building this as a solution that needs as little configuration and maintenance as possible and can be self hosted. While I have it running as a rough proof of concept it isn’t yet ready for publishing the code.

Since writing this blog post I’ve found out that for freelancers there is a cheaper way to work with Private Packagist so I don’t think anymore that a home-grown solution is worth the effort.

If you’re interested in this you may want to subscribe to my RSS feed or follow me on Twitter to not miss any updates on the matter.

🙏 Finally I’d like to thank Giuseppe Mazzapica for proofreading this post.