The daemon had a mechanism that allows it to handle a list of
substituters and try them sequentially; this removes it.
* nix/scripts/substitute.in: Remove.
* nix/local.mk (nodist_pkglibexec_SCRIPTS): Remove.
* config-daemon.ac: Don't output 'nix/scripts/substitute'.
* nix/libstore/build.cc (SubstitutionGoal)[subs, sub, hasSubstitute]:
Remove.
[tryNext]: Make private.
(SubstitutionGoal::SubstitutionGoal, SubstitutionGoal::init): Remove now
unneeded initializers.
(SubstitutionGoal::tryNext): Adjust to assume a single substituter: call
'amDone' upfront when we couldn't find substitutes.
(SubstitutionGoal::tryToRun): Adjust to run 'guix substitute' via
'settings.guixProgram'.
(SubstitutionGoal::finished): Call 'amDone(ecFailed)' upon failure
instead of setting 'state' to 'tryNext'.
* nix/libstore/globals.hh (Settings)[substituters]: Remove.
* nix/libstore/local-store.cc (LocalStore::~LocalStore): Adjust to
handle a single substituter.
(LocalStore::startSubstituter): Remove 'path' parameter. Adjust to
invoke 'settings.guixProgram'. Don't refer to 'run.program', which no
longer exists.
(LocalStore::querySubstitutablePaths): Adjust for 'runningSubstituters'
being a singleton instead of a list.
(LocalStore::querySubstitutablePathInfos): Likewise, and remove
'substituter' parameter.
* nix/libstore/local-store.hh (RunningSubstituter)[program]: Remove.
(LocalStore)[runningSubstituters]: Remove.
[runningSubstituter]: New field.
[querySubstitutablePathInfos]: Remove 'substituter' parameter.
[startSubstituter]: Remove 'substituter' parameter.
* nix/nix-daemon/guix-daemon.cc (main): Remove references to
'settings.substituters'.
* nix/nix-daemon/nix-daemon.cc (performOp): Ignore the user's
"build-use-substitutes" value when 'settings.useSubstitutes' is false.
Until now, something like "guix build sed -v1 --check" would not get a
'build-succeeded' event, which in turn meant that the spinner would not
be erased upon build completion.
* nix/libstore/build.cc (DerivationGoal::registerOutputs): When
'buildMode' is bmCheck and 'settings.printBuildTrace' emit a
"@ build-succeeded" trace upon success.
* tests/store.scm ("build-succeeded trace in check mode"): New test.
This makes it easier to run the uninstalled daemon.
* nix/local.mk (libstore_a_CPPFLAGS): Append "/guix" to
NIX_LIBEXEC_DIR.
* build-aux/pre-inst-env.in (NIX_LIBEXEC_DIR): Adjust comment.
* nix/libstore/builtins.cc (builtinDownload): Remove SUBDIR and its
use.
* nix/libstore/local-store.cc (runAuthenticationProgram): Ditto.
* nix/libstore/gc.cc (addAdditionalRoots): Remove "/guix" prefix.
* nix/nix-daemon/guix-daemon.cc (main): Ditto.
* nix/nix-daemon/nix-daemon.cc (performOp) <wopSetOptions>: When
'isRemoteConnection' is true, set 'settings.keepFailed' to zero.
* doc/guix.texi (Common Build Options): Document this behavior.
That way it is handled in the same way as other helper scripts.
* nix/scripts/guix-authenticate.in: Rename to...
* nix/scripts/authenticate.in: ... this.
* config-daemon.ac: Adjust accordingly.
* nix/local.mk (libstore_a_CPPFLAGS): Remove -DOPENSSL_PATH.
(nodist_libexec_SCRIPTS): Remove.
(nodist_pkglibexec_SCRIPTS): New variable.
* nix/nix-daemon/guix-daemon.cc (main): Remove 'setenv' call for
"PATH".
* nix/libstore/local-store.cc (runAuthenticationProgram): New function.
(LocalStore::exportPath, LocalStore::importPath): Use it instead of
'runProgram' and OPENSSL_PATH.
This allows clients to tell whether output comes from the daemon or, if
it comes from a builder, from which builder it comes. The latter is
particularly useful when MAX-BUILD-JOBS > 1.
* nix/libstore/build.cc (DerivationGoal::tryBuildHook)
(DerivationGoal::startBuilder): Print the child's PID in "@ build-started"
traces.
(DerivationGoal::handleChildOutput): Define 'prefix', pass it to
'writeToStderr'.
* nix/libstore/globals.cc (Settings:Settings): Initialize
'multiplexedBuildOutput'.
(Settings::update): Likewise.
* nix/libstore/globals.hh (Settings)[multiplexedBuildOutput]: New field.
Update 'printBuildTrace' documentation.
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0.163.
* nix/nix-daemon/nix-daemon.cc (performOp) <wopSetOptions>: Special-case
"multiplexed-build-output" and remove "use-ssh-substituter".
* guix/store.scm (set-build-options): Add #:multiplexed-build-output?
and honor it.
(%protocol-version): Bump to #x163.
* tests/store.scm ("multiplexed-build-output"): New test.
fixlet
This code has never been of any use in Guix.
* nix/libutil/archive.cc (useCaseHack): Remove.
(parse): Keep only the alternate branch in "if (useCaseHack)".
* guix/scripts/perform-download.scm (perform-download): Add
#:print-build-trace? and pass it to 'url-fetch'.
(guix-perform-download): Define 'print-build-trace?' and pass it to
'perform-download'.
* guix/build/download.scm (ftp-fetch): Add #:print-build-trace? and
honor it.
(url-fetch): Likewise.
* nix/libstore/builtins.cc (builtinDownload): Set _NIX_OPTIONS
environment variable.
* nix/libstore/build.cc (canBuildLocally): Allow building armhf-linux
builds on aarch64-linux.
(DerivationGoal::runChild) Throw error if attempting to build for
armhf-linux on an unsupported platform.
* doc/guix.texi (Invoking guix build): Document how to build natively
for armhf-linux on aarch64-linux. Add note that on some aarch64
machines this is unsupported.
* nix/nix-daemon/nix-daemon.cc (isRemoteConnection): New variable.
(performOp): For wopCollectGarbage, throw an error when isRemoteConnection
is set.
(acceptConnection): Set isRemoteConnection when connection is not AF_UNIX.
* tests/guix-daemon.sh: Add a test for the new behavior.
* config-daemon.ac: Don't bail out when libbz2 is missing. Define
'HAVE_LIBBZ2' Automake conditional.
* nix/libstore/build.cc: Wrap relevant bits in '#if HAVE_BZLIB_H'.
* nix/libstore/globals.cc (Settings::Settings): 'logCompression'
defaults to COMPRESSION_GZIP when HAVE_BZLIB_H is false.
* nix/libstore/globals.hh (CompressionType): Make 'COMPRESSION_BZIP2'
conditional on HAVE_BZLIB_H.
* nix/local.mk (guix_register_LDADD, guix_daemon_LDADD): Add -lbz2 only
when HAVE_LIBBZ2.
* nix/nix-daemon/guix-daemon.cc (parse_opt): Ignore "bzip2" when not
HAVE_BZLIB_H.
Fixes <https://bugs.gnu.org/29862>.
Reported by Danny Milosavljevic <dannym@scratchpost.org>.
* nix/scripts/list-runtime-roots.in (canonicalize-store-item): Define
'store' with a trailing "/". Have the 'string-prefix?' call match the
'string-drop' call.
This allows running as non-root.
Fixes a regression introduced in b8f59cdc20.
* nix/scripts/list-runtime-roots.in (referenced-files): Handle EACCES in
addition to ENOENT.
Looking at 'addAdditionalRoots' in libstore/gc.cc, it looks like it
should always have been this way. In practice it probably doesn't make
much of a difference.
* nix/scripts/list-runtime-roots.in (canonicalize-store-item): New
procedure.
<top level>: Use it.
This makes things a bit faster (0.8s instead of 1.4s on my laptop).
* nix/scripts/list-runtime-roots.in (lsof-roots): Remove.
(proc-fd-roots): Return the empty list when 'scandir' returns #f.
(referenced-files): New procedure.
Use it at the top level.
Prior to this change, errors such as wrong permissions on
/etc/guix/signing-key.sec would give:
guix-daemon: nix/libutil/serialise.cc:15: virtual nix::BufferedSink::~BufferedSink(): Assertion `!bufPos' failed.
This patch correctly propagates the error to the client and thus changes
that to:
error: build failed: file `/etc/guix/signing-key.sec' should be secret (inaccessible to everybody else)!
* nix/nix-daemon/nix-daemon.cc (performOp): Wrap 'exportPath' call in
'try' block.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Reported by Ricardo Wurmus.
* nix/nix-daemon/nix-daemon.cc (acceptConnection): Remove intermediate
'inaddr' variables that relied on implicit casts. This fixes
compilation with GCC 4.9.
* nix/nix-daemon/guix-daemon.cc (DEFAULT_GUIX_PORT): New macro.
(listen_options): New variable.
(parse_opt): Push back '--listen' options to LISTEN_OPTIONS.
(open_unix_domain_socket, open_inet_socket)
(listening_sockets): New functions.
(main): Use it. Pass SOCKETS to 'run'.
* nix/nix-daemon/nix-daemon.cc (matchUser): Remove.
(SD_LISTEN_FDS_START): Remove.
(acceptConnection): New function.
(daemonLoop): Rewrite to take a vector of file descriptors, to select(2)
on them, and to call 'acceptConnection'.
(run): Change to take a vector of file descriptors.
* tests/guix-daemon.sh: Add test.
Otherwise, users will be stuck running an old copy of guix and the guix-daemon
if they copy the service files instead of symlinking them.
* etc/guix-daemon.conf.in, etc/guix-daemon.service.in, etc/guix-publish.conf.in,
etc/guix-publish.service.in: Expand @localstatedir@ instead of @bindir@.
* nix/local.mk (etc/guix-%.service, etc/guix-%.conf): Use @localstatedir@
instead of @bindir@.
Reported by rohit yadav <rohityadav@utexas.edu>
at <https://lists.gnu.org/archive/html/guix-devel/2017-02/msg00191.html>.
This fixes a regression whereby 'guix perform-download' would always see
NIX_STORE as unset and thus use "/gnu/store", leading it to miscompute
the hydra.gnu.org content-addressed URLs when the store file name is not
"/gnu/store".
* nix/libstore/builtins.cc (builtinDownload): Add 'setenv' call for
'NIX_STORE'.
Fixes <http://bugs.gnu.org/20217>.
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0x161.
* nix/nix-daemon/nix-daemon.cc (performOp): "build-max-jobs",
"build-max-silent-time", and "build-cores" are no longer read upfront;
instead, read them from the key/value list at the end.
* nix/nix-daemon/guix-daemon.cc (main): Explicitly set
'settings.maxBuildJobs'.
* guix/store.scm (%protocol-version): Bump to #x161.
(set-build-options): #:max-build-jobs, #:max-silent-time, and
#:build-cores now default to #f. Adjust handshake to new protocol.
* tests/store.scm ("build-cores"): New test.
* tests/guix-daemon.sh: Add test for default "build-cores" value.
Fixes <http://bugs.gnu.org/25242>.
Reported by Leo Famulari <leo@famulari.name>.
The regression was introduced in 94d92c7796.
* nix/libstore/build.cc (DerivationGoal::startBuilder): Set 'useChroot'
as a function 'of isBuiltin(drv)'.
(DerivationGoal::runChild): Use 'useChroot' instead of
'useChroot && !isBuiltin(drv)'.
Fixes <http://bugs.gnu.org/15890>.
* nix/libstore/globals.hh (Settings) Add clientUid and clientGid.
* nix/nix-daemon/nix-daemon.cc (daemonLoop] Store UID and GID of the
caller in settings.
* nix/libstore/build.cc (_chown): New function.
(DerivationGoal::deleteTmpDir): Use it, change ownership of build
directory if it is kept and the new owner is not root.
Before that we'd have STDERR_WRITE round trips for very small amounts of
data, ranging from a few bytes for the metadata of nars to the size of
one file being exported.
With this change, something like:
guix archive --export /gnu/store/5rrsbaghh5ix1vjcicsl60gsxilhjnf2-coreutils-8.25 | dd of=/dev/null
reports a throughput of 35 MB/s instead of 25 MB/s before.
* nix/nix-daemon/nix-daemon.cc (TunnelSink): Inherit from 'BufferedSink'
rather than 'Sink'. Rename 'operator ()' to 'write'.
(performOp) <wopExportPath>: Add 'sink.flush' call.
* .gitignore: add etc/guix-publish.conf and /etc/guix-publish.service.
* etc/guix-publish.conf.in: New file.
* etc/guix-publish.service.in: New file.
* nix/local.mk (etc/guix-%.service, etc/guix-%.conf): Generalized former
build-rules for by using patterns.
(nodist_systemdservice_DATA): Add etc/guix-publish.service, update
comment.
(nodist_upstartjob_DATA): Add etc/guix-publish.conf, update comment.
* doc/guix.texi (Invoking guix publish): Add description for enabling
"guix publish" on host distros using the new files.
This solves a problem whereby if /gnu/store/.links had enough entries,
ext4's directory index would be full, leading to link(2) returning
ENOSPC.
* nix/libstore/optimise-store.cc (LocalStore::optimisePath_): Upon
ENOSPC from link(2), print a message and return instead of throwing a
'SysError'.
In particular, this eliminates a bunch of boilerplate code.
Also integrates these Nix commits:
80da7a6 Probably fix SQLITE_BUSY errors
37a337b throwSQLiteError(): Check for SIGINT so we don't loop forever
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Reported by Federico Beffa <beffa@ieee.org>
at <https://lists.gnu.org/archive/html/guix-devel/2016-05/msg00928.html>.
* nix/libstore/local-store.cc (LocalStore::querySubstitutablePaths)
(LocalStore::querySubstitutablePathInfos): Return when
'settings.useSubstitutes' is false.
* tests/store.scm ("references/substitutes missing reference info"):
Make sure to return #f on failure.
* tests/store.scm ("substitutable-path-info when substitutes are turned off"):
("substitutable-paths when substitutes are turned off"): New tests.
Previously files in the Nix store were owned by root or by nixbld,
depending on whether they were created by a substituter or by a
builder. This doesn't matter much, but causes spurious diffoscope
differences. So use root everywhere.
E.g.
$ nix-build pkgs/stdenv/linux/ -A stage1.pkgs.perl --check
nix-store: src/libstore/build.cc:1323: void nix::DerivationGoal::tryToBuild(): Assertion `buildMode != bmCheck || validPaths.size() == drv->outputs.size()' failed.
when perl.out exists but perl.man doesn't. The fix is to only check
the outputs that exist. Note that "nix-build -A stage1.pkgs.all
--check" will still give a (proper) error in this case.
If repair found a corrupted/missing path that depended on a
multiple-output derivation, and some of the outputs of the latter were
not present, it failed with a message like
error: path ‘/nix/store/cnfn9d5fjys1y93cz9shld2xwaibd7nn-bash-4.3-p42-doc’ is not valid
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Otherwise, since the call to write a "d" character to the lock file
can fail with ENOSPC, we can get an unhandled exception resulting in a
call to terminate().
* doc/local.mk: Use "%D%" for the directory of the fragment relative to
the base 'Makefile.am'.
* emacs/local.mk: Likewise.
* gnu/local.mk: Likewise.
* nix/local.mk: Likewise.
This follows a convention used by some other GNU packages like Autoconf,
Bison, Coreutils, and Gnulib.
* doc.am: Rename to ...
* doc/local.mk: ... this.
* emacs.am: Rename to ...
* emacs/local.mk: ... this.
* gnu-system.am: Rename to ...
* gnu/local.mk: ... this.
* daemon.am: Rename to ...
* nix/local.mk: ... this.
* Makefile.am: Adapt to them.
* doc/guix.texi (Porting to a New Platform): Adapt documentation.
* guix/config.scm.in (%state-directory, %config-directory): Adapt comments.
* emacs/guix-config.el.in (guix-config-state-directory): Likewise.
In Nix itself, the new 'BuildResult' type is returned by the new
'buildDerivation' method, which we don't have and need.
* nix/libstore/build.cc (Goal)[cancel]: Remove.
[timeOut]: New pure virtual method.
(DerivationGoal)[result]: New field.
[cancel]: Remove.
[timedOut, getResult, done]: New methods.
(DerivationGoal::cancel): Remove.
(DerivationGoal::timedOut): New method.
(DerivationGoal::haveDerivation): Call 'done' instead of 'amDone'.
(DerivationGoal::outputsSubstituted): Ditto.
(DerivationGoal::inputsRealised): Ditto.
(DerivationGoal::buildDone): Ditto.
(DerivationGoal::handleChildOutput): Call 'timedOut' instead of
'cancel'.
(DerivationGoal::done): New method.
(SubstitutionGoal)[cancel]: Remove.
[timedOut]: New method.
(SubstitutionGoal::cancel): Remove.
(SubstitutionGoal::timedOut): New method.
(Worker::waitForInput): Use it.
* nix/libstore/store-api.hh (BuildResult): New struct.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Rather than using $<host-TMPDIR>/nix-build-<drvname>-<number>, the
temporary directory is now always /tmp/nix-build-<drvname>-0. This
improves bitwise-exact reproducibility for builds that store $TMPDIR
in their build output. (Of course, those should still be fixed...)
* nix/libstore/build.cc (DerivationGoal)[tmpDirInSandbox]: New field.
(DerivationGoal::startBuilder): Initialize 'useChroot' earlier. Compute
'tmpDirInSandbox', and use it when populating 'dirsInChroot'.
* doc/guix.texi (Build Environment Setup): Document it.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
This makes it easy to detect non-deterministic builds.
* nix/libstore/build.cc (DerivationGoal): Remove 'InodesSeen'; add
'curRound', 'nrRound', and 'prevInfos'.
(DerivationGoal::inputsRealised): Initialize 'nrRound'.
(NotDeterministic): New error type.
(DerivationGoal::buildDone): Check whether we need to repeat.
(DerivationGoal::startBuilder): Adjust message.
(DerivationGoal::registerOutputs): Check whether we get the same result.
* nix/libstore/globals.cc (Settings::get(const string & name, int def)):
New method.
* nix/libstore/globals.hh (Settings): Add it.
* nix/libstore/store-api.hh (ValidPathInfo): Add operator ==.
* nix/nix-daemon/nix-daemon.cc (performOp): Allow "build-repeat" for
"untrusted" users.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0x10f.
* nix/libstore/remote-store.cc (RemoteStore::buildPaths): Send the
BUILDMODE when the daemon supports it. Reject invalid values of
BUILDMODE for old daemons.
* nix/nix-daemon/nix-daemon.cc (performOp) <wopBuildPaths>: Read the
build mode when the client supports it.
If automatic store optimisation is enabled, and a hard-linked file in
the store gets corrupted, then the corresponding .links entry will
also be corrupted. In that case, trying to repair with --repair or
--repair-path won't work, because the new "good" file will be replaced
by a hard link to the corrupted file. We can catch most of these cases
by doing a sanity-check on the file sizes.
Fixes https://github.com/NixOS/nixpkgs/issues/9504.
Note that this means we may have a non-functional /bin/sh in the
chroot while rebuilding Bash or one of its dependencies. Ideally those
packages don't rely on /bin/sh though.
