Toasty Deployer

I have somewhat reached the point where I have enough projects that deploying each of them has started to become a problem.

Which is, admittedly, a fairly good problem to have.

Between websites, Odoo modules, services, applications, and whatever else I need to build, there are now quite a few repositories that need something to happen after I push the changes to GitHub.

• Some need to be deployed to my servers.
• Some need to be built or compiled.
• Some need release files generated.
• Some need GitHub releases, or Google Play, or Microsoft Store releases etc.,.

And most require multiple of those.

My first thought had been to tackle it all with GitHub Actions.

But then I realised I'm crap at them and creating individual workflows on each repo to do the same things, in basically the same ways, was just getting tedious.

Instead I built my own simple solution called the Toasty Deployer. There's no need for GitHub Actions in the repo, no extra setup to do with config files. Just 2 simple things, a deploy script on the server, and a reason to call it.

For example, if I push to:

All-Things-Toasty-Software-Ltd/bakers-archive-mobile-companion

It will look for and run:

/repos/All-Things-Toasty-Software-Ltd/bakers-archive-mobile-companion.sh

And that's it. If I create a new repo, it will just look for a script in that format. If one doesn't exist, it'll just skip, and if one does, it will run it, directly on my servers, with the access to easily deploy it in production environments.

What the Script Does Is Up to the Project

One of the things I like most about this approach is that it doesn't need to understand how each project works. It doesn't need to explicitly have support for Kotlin, or Odoo, or whatever, it just needs to identify the repository and run a script.

The server already has any dependencies needed, so i dont need to deal with setting up a new environment each time, and waiting for every package to be installed, I can just run the script, and move on. And if a different service doesn't need it, it doesn't affect it.

It's Also a Proper GitHub App

I didn't just want this to be a random webhook endpoint sitting on a server and blindly executing commands whenever something sends it JSON.

That probably wouldn't end well, and so instead Toasty Deployer is implemented as a GitHub App and verifies incoming webhook payloads using GitHub's HMAC signature.

When a push arrives, the service verifies that the request is legitimate before doing anything else.

After that, it responds and moves the actual deployment into a background worker. That worker can then:

1. Create a deployment record.
2. Generate an installation token for the GitHub App.
3. Create a GitHub Check Run
4. Run the repository deployment script
5. Captures the output from the process.
6. Store the results and logs.

I Wanted the Logs Outside GitHub Too

Another part of the project that I particularly wanted was its own record of deployments.

GitHub knows about it, but I wanted my own systems to be able to ask questions like:

• What repositories have deployed recently?
• What was the latest deployment for this project?
• Did it succeed?
• What commit triggered it?
• What did the deployment actually output?

So Toasty Deployer stores deployment runs in SQLite and exposes them through a REST API.

That means something else can ask for the latest deployment for a repository and get structured information back.

This is where the project starts connecting rather nicely with the rest of my software.

Because, naturally, one of the things I immediately thought about was Odoo.

An Odoo module can query Toasty Deployer and show deployment information inside Odoo itself. Repository status, commit information and even deployment logs can become part of the wider system rather than living exclusively inside GitHub.

Apparently, when you build enough infrastructure, eventually you start building infrastructure to monitor your infrastructure.

I suspect this is how it begins.

The Bit I'm Most Interested in Using

The biggest advantage for me isn't necessarily that Toasty Deployer is technically more complicated than GitHub Actions.

In many ways, the appeal is the opposite.

I don't have to remember the exact structure of a workflow every time I create a new project.

I don't need to add server connection details and deployment boilerplate to every repository.

I install the GitHub App, give Toasty Deployer access to the repositories I want it to handle, and put the deployment instructions on the deployment server.

Then, when I push:

GitHub
  ↓
Toasty Deployer
  ↓
Find matching script
  ↓
Run the script
  ↓
Report what happened

Nice. Simple. Toasted.

It's Early, But It's Already Useful

Toasty Deployer is very new, at the time of writing, I am still actively developing it. But it has already become something that I'm using everywhere.

It started with a very specific annoyance: I have multiple projects, multiple deployment requirements, and I didn't particularly want to maintain a nearly identical GitHub workflow in every single repository.

The answer, naturally, was to build another service.

Because apparently my response to having too much software to manage is to write more software to manage it.

But this one feels useful.

Toasty Deployer is now publicly available as an open-source project under the LGPLv3 licence and is owned by All Things Toasty Software Ltd.

You can find the project on GitHub or read the docs at Toasty Software.

And, assuming everything works correctly, every future push I make should hopefully mean one less thing I have to remember to deploy manually.

That alone is worth building a little more infrastructure for.