Getting Started¶

Register¶

Registering for nixbuild.net is simple, and every account includes 50 free CPU hours of build time, so you can try out the service on your own terms, without committing to anything.

Before filling out the registration form, you need to generate a new SSH key for authenticating with the service. Once registered, you can add and remove SSH keys as you like.

The SSH key must be of type Ed25519. With OpenSSH, you can generate the key in the following way:

ssh-keygen -t ed25519 -C my-key-comment -f my-nixbuild-key

If you're using nix-daemon to run nix builds (which is usually the case), you shouldn't set a password on the SSH key. The key comment provided with the -C option will help you distinguish keys within nixbuild.net, if you add multiple SSH keys to your account, but is not used in any other way.

Now you can fill out the registration form, providing the public part of your SSH key. The public key should have this syntax:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDzLYWrTuAlSWsdTHkTwLGoIaXOq6vrifbBt/X060KwL my-key-comment

When you've submitted the registration form, an activation link is emailed to you. After you've activated the account, follow the rest of this guide in order to setup your system for using nixbuild.net.

If you run into any troubles when going through the guide, or during your future use of nixbuild.net, just ask [email protected] for help.

You are also welcome to submit issue reports, questions or any other feedback here.

SSH Configuration¶

If you use nix-daemon to run builds on your local machine you need to make the private SSH key available (with permissions 0600) to the user that nix-daemon runs as (root). The easiest way to do this is to add the following entry to your system wide (or root user) SSH client config:

Host eu.nixbuild.net
  PubkeyAcceptedKeyTypes ssh-ed25519
  IdentityFile /path/to/your/private/key

The PubkeyAcceptedKeyTypes ssh-ed25519 entry above is important, since it will keep SSH from using key types that nixbuild.net can't handle.

Then, you need to add the public host key of eu.nixbuild.net to the known hosts file of the root user (/root/.ssh/known_hosts). Alternatively, you can add it to the global known hosts file (/etc/ssh/ssh_known_hosts). The reason you have to add the public host key manually is that when Nix sets up an SSH connection, it can't ask you to confirm the host key.

You should add the following line to the known hosts file:

eu.nixbuild.net ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPIQCZc54poJ8vqawd8TraNryQeJnvH1eLpIDgbiqymM

Verifying Shell Access¶

You should now verify that you can connect to the nixbuild.net shell, which is used to administer your account.

Run the following command (using the root user via sudo, to emulate how nix-daemon works):

sudo ssh eu.nixbuild.net shell

You should be greeted by a screen looking similar to this:

••••••••••••••••••••••••••••
••••••• nixbuild.net •••••••
••••••••••••••••••••••••••••

Welcome to nixbuild.net!

This shell allows you to administer your account
and retrieve information about your nix builds.

Account name: bob
Free build time left: 49:59:59

Available shell commands:

  help       Show help
  get        Show all account settings
  set        Set an account settings
  unset      Revert an account setting to default
  list       List entities
  show       Show details about entities
  usage      Show resource usage
  ssh-keys   Manage your public SSH keys

Run 'help COMMAND' for help on individual shell commands.
For help on subcommands run 'COMMAND SUBCOMMAND --help'.
For more documentation, see:
https://docs.nixbuild.net/nixbuild-shell

Adding an SSH Key¶

For convenience, you probably want to generate another SSH key and add it to your nixbuild.net account now. This way, you can have one key owned by the root user, used by nix-daemon, and one key owned by your normal user, for accessing the nixbuild.net shell.

Your user key can have a password or be managed by an SSH agent, since it will be used interactively. It must be of type Ed25519, and you need to have the PubkeyAcceptedKeyTypes ssh-ed25519 configured as above.

Each key associated with your account must have a unique key name (sometimes referred to as key comment) for identifying the key.

To add a new SSH key, use the ssh-keys add command, like this:

nixbuild.net> ssh-keys add --help
ssh-keys add - Add a public SSH key to your account

Usage: ssh-keys add KEY_TYPE KEY KEY_NAME
  Add a key

Available options:
  KEY_TYPE                 Type of SSH key (must be 'ssh-ed25519')
  KEY                      The base64-encoded key byte sequence
  KEY_NAME                 A key name (comment), for identifying keys
  -h,--help                Show this help text

Example usage:
  ssh-keys add ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDrqgdPd6EB9YOUPy5CRNc+3dDvkc5jc2ZXfBNusi3LF laptop

The options match the format of OpenSSH public keys.
Only SSH keys of type 'ssh-ed25519' are supported.
Each key in your account must have a unique key name.
The same key cannot be associated with more than one account.


WARNING! When adding a new public key, any person with access to the
corresponding private key will gain immediate access to this account, and will
be able to run builds and make changes to the account. Look into the settings
'allow-shell' and 'allow-override' to restrict permissions.

As mentioned in the warning printed above, new SSH keys will get full access to your account by default. You can use various settings to restrict the key, see for example allow-shell and allow-override.

You can add as many SSH keys as you like to your account, to set up for example team users or CI builders.

Nix Configuration¶

You can now go ahead and configure nixbuild.net as a Nix remote build machine. As such, you can add it as an entry in your /etc/nix/machines file:

eu.nixbuild.net x86_64-linux - 100 1 big-parallel,benchmark

or simply use the --builders option of nix-build:

--builders "ssh://eu.nixbuild.net x86_64-linux - 100 1 big-parallel,benchmark"

The big-parallel,benchmark assignment is something that is called system features in Nix. You can use that as a primitive scheduling strategy if you have multiple remote machines. Nix will only submit builds that have been marked as requiring a specific system feature to machines that are assigned that feature.

The number 100 in the file above tells Nix that it is allowed to submit up to 100 simultaneous builds to eu.nixbuild.net. Usually, you use this property to balance builds between remote machines, and to make sure that a machine doesn't run too many builds at the same time. This works OK when you have rather homogeneous builds, and only one single Nix client is using a set of build servers. If multiple Nix clients use the same set of build servers, this simplistic scheduling breaks down, since a given Nix client loses track on how many builds are really running on a server.

However, when you're using nixbuild.net, you can set this number to anything really, since nixbuild.net will take care of the scheduling and scaling on its own, and it will not let multiple Nix clients step on each other's toes.

You probably want to activate the builders-use-substitutes Nix option. This option allows nixbuild.net to download dependencies directly from cache.nixos.org. You can read more about how nixbuild.net uses binary caches here.

Your First Build¶

Run the following build to verify that your configuration works as expected:

nix-build \
  --max-jobs 0 \
  --builders "ssh://eu.nixbuild.net x86_64-linux - 100 1 big-parallel,benchmark" \
  -I nixpkgs=channel:nixos-20.03 \
  --expr '((import <nixpkgs> {}).runCommand "test${toString builtins.currentTime}" {} "echo Hello nixbuild.net; touch $out")'

The output should look something like this:

these derivations will be built:
  /nix/store/0alhvbxdqq4hakna5sp6l51180q2l1l9-test1598214851.drv
building '/nix/store/0alhvbxdqq4hakna5sp6l51180q2l1l9-test1598214851.drv' on 'ssh://eu.nixbuild.net'...
Hello nixbuild.net
copying 1 paths...
copying path '/nix/store/8zj14ysp2jd3whldi99hk501628a9gaf-test1598214851' from 'ssh://eu.nixbuild.net'...
/nix/store/8zj14ysp2jd3whldi99hk501628a9gaf-test1598214851

That's it, now you can use nixbuild.net!

Running Distributed Builds¶

Read through the Nix remote build documentation to get an understanding for how Nix performs remote builds. Everything there applies to nixbuild.net, since nixbuild.net just appears as an ordinary remote builder to Nix.

When setting up remote builds it can be helpful to use the --max-jobs 0 Nix option, that explicitly disables local builds. That way, if remote builds are not working for some reason, this will not be obscured by local building.

Nix prefers using remote builders before building locally. If you want to disable remote builds for a specific build session, you can provide nix-build with the option --builders ''.

NixOS configuration¶

NixOS users can use a configuration similar to the one below to configure both SSH and Nix distributed builds:

programs.ssh.extraConfig = ''
  Host eu.nixbuild.net
    PubkeyAcceptedKeyTypes ssh-ed25519
    IdentityFile /path/to/your/private/key
'';

programs.ssh.knownHosts = {
  nixbuild = {
    hostNames = [ "eu.nixbuild.net" ];
    publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPIQCZc54poJ8vqawd8TraNryQeJnvH1eLpIDgbiqymM";
  };
};

nix = {
  distributedBuilds = true;
  buildMachines = [
    { hostName = "eu.nixbuild.net";
      system = "x86_64-linux";
      maxJobs = 100;
      supportedFeatures = [ "benchmark" "big-parallel" ];
    }
  ];
};

If you use the above configuration you don't have to set up your known_hosts or .ssh/config files manually. You only need to put your private SSH key in place and assign it correct permissions.