Flakes for production and development
Recently, I took a deeper look into Nix flakes. I won’t go into detail about basics in this blog post. To be ready to understand when reading further, I assume some developer experience and also some basic NixOS knowledge, thus don’t expect an introduction to flakes or nix/NixOS itself.
Still with me? Perfect. During my research, I was particular interested how to leverage them as build tool, if they can have an impact on development workflows, and reproducibility during the usual engineering cycle. I’d like to share my experience during my experiments.
Kinda in reverse order, starting with the conclusion: This is fantastic, why did I miss that before?! They have a huge potential to accelerate development workflows, boost reproducibility and make our engineering lives easier. Wherever possible, I am going to push for it, at least for my small private projects. Though, I am unsure if they should be a first-class citizen for everything. Gradually changing existing build processes and development flows can be quite time consuming. For new projects it might be worth a try though. Also, I have no experience how this works out in distributed teams with members using that Micro*** OS (which should work just fine).
Development workflow
Collaborating with peers or on your personal projects can be cumbersome when these projects require a lot of different tooling. Maybe you’re a front-end developer, some projects use yarn, some pnpm, some are still on Node 20, and some are on recent Node LTS already. When frequently switching between (very different) projects, you’ll likely end up with a lot of tools on your PATH or some kind of tool to manage those tools. Just to cover every project. Version conflicts might pop up, specific combinations don’t work, or worse, you need to manually dive into bootstrapping tools just for specific projects, adjusting what’s on your PATH or in your environment and change every time.
I learned that there’s really no need to. When I came across direnv (and it’s actually quite popular for some time), I noticed for myself: You did it wrong all the time.
Instead of preparing my operating system for the projects I work on, the projects should prepare my OS/configuration for me to work on them.
The combination of direnv with flakes is convenient. Probably no other stack can provide similar guarantees to have identical and reproducible development and production (more on that later) setups.
How does it work? How can we get the same experience on any machine with nix installed without messing up our PATH or installing tools to manage tools?
We’ll use flakes in conjunction with a tool called direnv (or better the improved nix variant nix-direnv which can handle the use flake directive). We’ll not go into details how you can set them up properly, but home manager makes it easy to get started on your machine.
Once required tools are ready (nix and nix-direnv/direnv), let’s dive deeper into our new project which we call myapp which resides on our disk at ~/Workspaces/myapp/. We plan to have bootstrap a static site generator project with gohugo.
Bootstrap your project using nix flakes
Everything (good) begins with a flake.nix file in the root project folder. It defines the environment we like to set up once we change directory into the project folder with the so called devShells (development shells).
1# flake.nix
2{
3 description = "A basic flake with a dev shell";
4 inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-24.11";
5 inputs.systems.url = "github:nix-systems/default";
6 inputs.flake-utils = {
7 url = "github:numtide/flake-utils";
8 inputs.systems.follows = "systems";
9 };
10
11 outputs = { nixpkgs, flake-utils, ... }:
12 flake-utils.lib.eachDefaultSystem (system:
13 let pkgs = nixpkgs.legacyPackages.${system};
14 in {
15 devShells.default =
16 pkgs.mkShell { packages = [ pkgs.hugo ]; };
17 });
18}We also need a .envrc file with use flake as content.
1# .envrc
2use flakeLet’s create a lock file. It controls the flake versions we get for any dependency. Invoke nix flake lock which creates a flake.lock file. Make sure to version control it. I also recommend adding .direnv to your .gitignore.
Let’s go back to our application. We want to create a web site which uses the static site generator gohugo. Obviously, we need the hugo binary. Usually, you would download it or install it into your PATH. You also need to think about updating it periodically. Not anymore! Everything’s ready to use with our flake definition packages = [ pkgs.hugo ]. Change directory into ~/Workspaces/myapp. If you allow direnv to be invoked (it will ask for it), then you should see an indicator in your shell. hugo is available. You can start using it as it would be globally on your PATH.
This is how it could look like:
1direnv: loading ~/Workspaces/myapp/.envrc
2direnv: using flake
3direnv: nix-direnv: Using cached dev shell
4direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +HOST_PATH +IN_NIX_SHELL +LD +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_BUILD_CORES +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_CFLAGS_COMPILE +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_LDFLAGS +NIX_STORE +NM +OBJCOPY +OBJDUMP +RANLIB +READELF +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +__structuredAttrs +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +mesonFlags +name +nativeBuildInputs +out +outputs +patches +phases +preferLocalBuild +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system ~PATH ~XDG_DATA_DIRS
5
6~/Workspaces/myapp master*
7nix-shell-env ❯ which hugo
8/nix/store/5h5q5jjr64chslxp4qwjqn86vzc1ybj5-hugo-0.136.5/bin/hugoWhat we’ve achieved: an environment based on the directory you’re in. If you have a lot of projects, switching environments has never been easier. No mess on your PATH anymore. The project decides what you need and what everyone gets.
Even if you only have few projects to work on, using this method (maybe we call it git native flakes repository?) makes it easy for anyone to get started. Your environment is “inside your directory” and automatically started. You can add any nix package to it, even specific Python dependencies, Ruby, or specific versions of NodeJS and npm, e.g., with packages = [ pkgs.nodejs_20 pkgs.pnpm_9 ];.
Key takeaways from this section:
- Use flakes to manage your required environments and easily switch between them through convenience feature of
direnv(nix-direnv) - Identical development setup for everyone
- Easy to get started, just needs
nixandnix-direnv/direnvon the system - Easy to maintain and upgrade with
nix flake update(and look into nix’inputsto follow nix releases) and everyone gets the new required environment once they change into a project’s directory - Huge time saver: you don’t need to update your dependencies on your system for all the different tools you need. The flake is doing it for you and for everyone else involved in your project as well
The static web site example is kinda simple and might not capture what you’re looking for. It also doesn’t cover the power of flakes, though it clearly shows that you can easily set up required environment/packages through flakes which are then used as development shells using direnv. Any developer in your team only needs to properly set up nix as package manager and have direnv installed. “It works on my machine” is no excuse anymore. If it works on your machine, it will work on production (and vice versa), because we’ll make production artifacts leverage our flakes as well.
Packaging and shipping
Packaging and shipping your application to production is mostly done as container image or natively. Let’s touch on the container approach first.
You might be familiar that Dockerfile container images can be quite tedious to create in the first place (... AS builder) to avoid unnecessary layers. It gets worse when we think about long-term maintenance and horrible when we talk about security. You often end up with stuff in your image you actually don’t need. Also, your development setup is likely different, right? Some packages from a repository of your operating system or maybe you’ve just downloaded that binary and put it into your PATH and now it simply “works”? And a peer wrote this pipeline with some other versions. Still works? But, should it “work” like that? The answer to that question is probably no, it should not, we should do better.
If you think about container images, you most likely want to use FROM scratch to reduce unnecessary bloat and attack surface. The issue with only adding your stuff to the image (even with builder) falls short the moment you require different types of linking and libraries. It becomes a mess to manage. Then, your next choice might be FROM alpine.
Compared to scratch images, we can achieve similar results with flakes. We already have our flake.nix file in the project’s root directory and use it for our local development setup. Let’s enhance that to build an example Go application a container image from that binary.
Here’s the full flake.nix:
1{
2 description = "A basic flake with a dev shell";
3 inputs = {
4 nixpkgs.url = "github:nixos/nixpkgs/nixos-24.11";
5 systems.url = "github:nix-systems/default";
6 flake-utils = {
7 url = "github:numtide/flake-utils";
8 inputs.systems.follows = "systems";
9 };
10 };
11
12 outputs = { nixpkgs, flake-utils, ... }:
13 flake-utils.lib.eachDefaultSystem (system:
14 let pkgs = nixpkgs.legacyPackages.${system};
15 in {
16 packages = rec {
17 default = pkgs.buildGoModule {
18 pname = "myapp";
19 version = "latest";
20 pwd = ./.;
21 src = ./.;
22 CGO_ENABLED = 0;
23 # input actual hash by building once locally
24 vendorHash = "sha256-AAAA";
25 };
26 container = pkgs.dockerTools.buildImage {
27 name = "myapp";
28 tag = "latest";
29 created = "now";
30 copyToRoot = pkgs.buildEnv {
31 name = "image-root";
32 paths = [ default ];
33 pathsToLink = [ "/bin" ];
34 };
35 config.Cmd = [ "${default}/bin/myapp" ];
36 };
37 };
38 devShells.default =
39 pkgs.mkShell { packages = with pkgs; [ go ]; };
40 });
41}Run nix build to build the native application (it defaults to target default). It will be placed into a result/ folder. Make sure you also add this directory/file to your .gitignore.
To build your container image, invoke nix build .#container. It will automatically build the defined default package and use it. Then, import it into your image registry with docker load < result (or podman) to make it available for further usage.
For one of my projects, I gave it a shot. It currently uses the AS builder approach and the main container image is derived with FROM alpine. Compared to the traditional way, the produced image by nix flakes is only ~71% in size.
1❯ podman image ls
2REPOSITORY IMAGE ID SIZE
3git.myservermanager.com/varakh/upda 0925c69fcf0f 48.8 MB
4withnixflakes/varakh/upda e574f8840f1d 34.9 MBReproducibility
When you first invoke nix build, you’ll see that it complains about a mismatched vendorHash. You can paste the proposed hash of the output into the flake.nix file. Subsequent invocations then succeed. If you don’t change your application afterward, outputs stay the same, thus the hash stays the same. Locally and on any build pipeline. If you’ve missed updating the hash, then your build fails. Transposing such a produced artifact through your different development, staging and production environments is pretty straight forward (just remember to use the load command above). Certainly, there are some benefits we’ve gained with this:
- The artifact is 100% identical when you build it, independent of your environment (due to the hash).
- The flake’s lock file ensures development and production gets the same tooling.
direnv/nix-direnvprovides a convenient way to develop your application on any machine.- Produced (container image) artifacts are smaller in size compared to the traditional way with your
Dockerfile, thus they implicitly reduce attack surface. There’s no 3rd party dependency in your image. No alpine, no ubuntu. You’re not bound to upstream changes. It’s like theFROM scratchapproach, but way easier to get started and to maintain.
Build pipeline
When switching to nix (flakes) as primary build and development tool, your pipeline needs to change. Good news is, that you only need nix and enable flake usage with NIX_CONFIG=experimental-features = nix-command flakes there. Not more. Pretty straight forward and the same as locally.
Bad news is, that you might encounter increased disk space usage due to the nix building steps. Make sure that pipeline executors are deleted or at least clean up their nix store periodically. Furthermore, to do this at large scale (and I don’t have any experience with that), you probably want add your own nix cache to avoid downloading everything all over again.
Trying this out in my ecosystem
For all my private project, I use Forgejo and their runner concept. Depending on a label you select, a runner picks up a job and executes it after you’ve registered it to your instance. These labels are prefixed with docker, host, or alike. From these labels, they derive the environment they execute the pipeline in.
What we need is a “nix” runner then. As all of my machines are on NixOS, that’s pretty straight forward. I set up a new Forgejo Runner on NixOS with NixOS Containers to ensure they’re isolated.
1{ ... }: {
2 containers.gitea-runner = {
3 autoStart = true;
4 privateNetwork = false;
5 config = { config, pkgs, ... }: {
6 networking.hostName = "my-native-runners";
7 networking.firewall.enable = true;
8 environment.systemPackages = with pkgs; [ ];
9 services.gitea-actions-runner = {
10 package = pkgs.forgejo-actions-runner;
11 instances.native-runner = {
12 enable = true;
13 name = "runner-container";
14 url = "https://forgejo.domain.tld";
15 labels = [ "native-container:host" ];
16 hostPackages = with pkgs; [
17 bash
18 coreutils-full
19 curl
20 gawk
21 gcc
22 gitMinimal
23 gnumake
24 gnused
25 gnutar
26 gzip
27 nix
28 nix-direnv
29 podman
30 wget
31 ];
32 settings = {
33 log = { level = "info"; };
34 runner = {
35 capacity = 1;
36 timeout = "1h";
37 insecure = false;
38 fetch_timeout = "10s";
39 fetch_interval = "10s";
40 };
41 cache = { enabled = false; };
42 };
43 };
44 };
45 system.stateVersion = "24.11";
46 };
47 };
48}Then, with the applications git repository already having the flake.nix, we need to change the step definition the runner executes in the build.yaml workflow file:
1on:
2 push:
3 branches:
4 - master
5 pull_request:
6 types: [ opened, synchronize, reopened ]
7jobs:
8 build:
9 runs-on: native-container
10 steps:
11 - uses: actions/checkout@v3
12 name: Checkout
13 - name: Build and test
14 shell: bash
15 run: |
16 nix build
17 nix build .#container Let me know what you think. The aspects of reproducibility, developer convenience, reducing attack surface, and the simplicity in pipelines are really convincing to me. As I don’t have the setup running for a long time, I cannot derive any conclusion on maintenance costs yet, but at least from a complexity perspective it seems to just “bump the flake” with nix flake update and adapt it if changed. But you only need to do that once, not distributed in all your build tools and (local) environments.
Side note: If you don’t want to dive into nix flakes and its ecosystem entirely, you’re probably missing out on something very cool, but nevertheless, direnv plays nicely for other directives like dotenv in your .envrc which ensures that environment variables from a project’s .env file are available once you enter the project’s directory. This could already provide value to automate and start your projects in seconds. No more tinkering with your environment.