Getting Started

Generate Authentication Key

If you've been granted access to nixbuild.net, you must generate a new ssh key of type Ed25519 to use for authenticating with nixbuild.net. With OpenSSH, you can generate the key in the following way:

ssh-keygen -t ed25519 -f my-nixbuild-key

If you're using nix-daemon to run nix builds, you shouldn't set a password on the ssh key.

Provide [email protected] with the public part of your new ssh key, so it can be registered to your account.

Configure Nix

If you use nix-daemon to run builds on your local machine you need to make the private ssh key available (with permissions 0600) to the user that nix-daemon runs as (usually root). The easiest way to do this is to add the following entry to your system wide (or root user) ssh client config:

Host beta.nixbuild.net
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null
  PubkeyAcceptedKeyTypes ssh-ed25519
  IdentityFile /your/private/key

The StrictHostKeyChecking no, UserKnownHostsFile /dev/null is there because currently the nixbuild.net host key will be re-generated from time to time. This will of course be fixed in the future.

You can now go ahead and use nixbuild.net as a Nix remote build machine. As such, you can add it as an entry in your /etc/nix/machines file:

beta.nixbuild.net x86_64-linux - 100 1 big-parallel,benchmark

or, using nix build or nix-build with the --builders option:

--builders "ssh://beta.nixbuild.net x86_64-linux - 100 1 big-parallel,benchmark"

The big-parallel,benchmark assignment is something that is called system features in Nix. You can use that as a primitive scheduling strategy if you have multiple remote machines. Nix will only submit builds that have been marked as requiring a specific system feature to machines that are assigned that feature.

The number 100 in the file above tells Nix that it is allowed to submit up to 100 simultaneous builds to beta.nixbuild.net. Usually, you use this property to balance builds between remote machines, and to make sure that a machine doesn't run too many builds at the same time. This works OK when you have rather homogeneous builds, and only one single Nix client is using a set of build servers. If multiple Nix clients use the same set of build servers, this simplistic scheduling breaks down, since a given Nix client loses track on how many builds are really running on a server.

However, when you're using nixbuild.net, you can set this number to anything really, since nixbuild.net will take care of the scheduling and scaling on its own, and it will not let multiple Nix clients step on each other's toes. In fact each build that nixbuild.net runs is securely isolated from other builds and by default gets exclusive access to the resources (CPU and memory) it has been assigned.

Verify Your Configuration

Run the following dummy build to verify that your configuration works as expected:

nix-build \
  --max-jobs 0 \
  --builders "ssh://beta.nixbuild.net x86_64-linux - 100 1 big-parallel,benchmark" \
  -I nixpkgs=channel:nixos-20.03 \
  --expr '((import <nixpkgs> {}).runCommand "test" {} "echo BUILDING; touch $out")'

The build should just print BUILDING and create an empty file as its output. You will also see some output from the nixbuild.net service.

Running Distributed Builds

Read through the Nix remote build documentation to get an understanding for how Nix performs remote builds. Everything there applies to nixbuild.net, since nixbuild.net just appears as an ordinary remote builder to Nix.

When setting up remote builds it can be helpful to use the --max-jobs 0 Nix option, that explicitly disable local builds. That way, if remote builds are not working for some reason, this will not be obscured by local building.

Nix prefers using remote builders before building locally. If you want to disable remote builds for a specific build session, you can provide nix-build with the option --builders ''.

NixOS configuration

NixOS users can use a configuration similar to the one below to configure both ssh and Nix distributed builds:

programs.ssh.extraConfig = ''
  Host beta.nixbuild.net
    StrictHostKeyChecking no
    UserKnownHostsFile /dev/null
    PubkeyAcceptedKeyTypes ssh-ed25519
    IdentityFile /your/private/key
'';

nix = {
  distributedBuilds = true;
  buildMachines = [
    { hostName = "beta.nixbuild.net";
      system = "x86_64-linux";
      maxJobs = 100;
      supportedFeatures = [ "benchmark" "big-parallel" ];
    }
  ];
};