2018-11-06

<clever> since the .. entries are technically hardlinks
<clever> Taneb: the hardlink count for trash, is based on the number of subdirs it has
<clever> Taneb: you might even get away with `ls -lhd /nix/store/trash`
<clever> exarkun22: if you also change the name in there, does the error show the new name?
<clever> exarkun22: yeah, that should be working
<clever> Taneb: ah, about all you can do there is to `watch du -h --max=0 /nix/store/trash` or fix nix to give more progress
<clever> Taneb: you can re-run it several times, or just give it a bigger number
<clever> exarkun22: oh, how are you doing an override to add it to testHaskellDepends ?
<clever> Taneb: then it just stops after finding 2g worth of garbage
<clever> Taneb: you can also make it somewhat faster via `nix-collect-garbage --max-freed 2g`
<clever> Taneb: yeah
<clever> exarkun22: if doCheck is false, it will filter all test deps out automatically
<clever> exarkun22: is doCheck = true; also set?
<clever> and when its done collecting garbage, all the garbage is in the trash folder, literally
<clever> directory moves are atomic
<clever> Taneb: to make deletion of /nix/store/a_directory/ atomic, it doesnt actually delete it, it just moves it into trash
<clever> which happens to be the exact tool that creates Taneb's gc roots!
<clever> nDuff: i have actually crashed my machine when i did that with a hydra internal tool
<clever> nDuff: so it doesnt flood /nix/store with 10,000 .drv files
<clever> nDuff: i suspect nix-env is operating in a read-only manner
<clever> Taneb: oops, ^^^
<clever> TweyIII: that depends on the value of `number of evaluations to keep` on https://hydra.iohk.io/jobset/serokell/iohk-ops#tabs-configuration
<clever> nDuff: anything like chroots going on?
<clever> nDuff: nix-store --delete the path it had downloaded
<clever> nDuff: nix-env -qavvvvvv
<clever> TweyIII: howoldis
<clever> nDuff: are you able to read recurseForDerivations from nix repl?
<clever> nDuff: so if the keys on that set depend on a download, it must download to read .recurseForDerivations
<clever> nDuff: all keys in a set must be known before you can do anything in that set
<clever> it even looks identical to several USB sticks i have
<clever> real nasty thing
<clever> yeah, ive seen that
<clever> i tracked it down to a motherboard driver for cellphone charging
<clever> ive managed to BSOD my win7 machine with a teensy microcontroller
<clever> sphalerite: sounds like the embeded controller locked up
<clever> nixos-rebuild --option substituters "http://cache.earthtools.ca local?root=/olddrive/" i think
<clever> ah, that also works
<clever> as long as the drive is equal or bigger, you can just clone the disk, thats the simplest one
<clever> nix copy can also be used, but then all non-nix data (including configuration.nix and $HOME) wont be copied
<clever> tathougies: if you just dd the entire disk (sda -> sdb for ex), it will also clone the partition tables for you
<clever> tathougies: if you just clone the entire thing with dd or ddrescue, then it should just boot
<clever> sphalerite: there is systemd-run, to launch things as a child of systemd, but it wont set all of the env vars
<clever> > builtins.toJSON (lib.foldl' lib.recursiveUpdate {} [ { a.b.c = 1;} { a.b.d = 2; } ])
<clever> > lib.foldl' lib.recursiveUpdate {} [ { a.b.c = 1;} { a.b.d = 2; } ]
<clever> > foldl' recursiveUpdate {} [ { a.b.c = 1;} { a.b.d = 2; } ]
<clever> nice
<clever> you could look at how the nixos ones work, and maybe reuse them
<clever> yeah
<clever> so setuid just doesnt work outside of nixos
<clever> but there is nothing setup to run something from nix, as root, when the machine boots
<clever> all setuid stuff is handled by wrappers made in /run/wrappers/bin/ when nixos boots
<clever> colemickens: nix doesnt allow setuid files in /nix/store/
<clever> stepcut: but i already have unencrypted keys for build slaves
<clever> stepcut: in my case, i'm using gpg-agent for my ssh agent, and its socket is at a predictable place, so its a bit simpler if i wanted to do that
<clever> thomasjm: it should detect that some "invalid" storepaths are in the cache, and just ignore the cache
<clever> thomasjm: so thats a different bug, but should still be filed to nix
<clever> thomasjm: and nix then complains that those /nix/store/ paths dont exist
<clever> thomasjm: you missed ~/.cache/nix/ which is pointing into /nix/store/
<clever> ottidmes: i sometimes do, but it has been broken on nixos at times
<clever> thomasjm: i think the cache is to blame here, different bug, what is the contents of ls -lh home/tom/.cache/nix/tarballs/071qfgdy82dkn58wqsv1ri3zlch4amvvfgn5vmwh9fvb4spcv18y-file
<clever> thomasjm: oh wait, was nix previously used on this, without that change?
<clever> thomasjm: definitely looks like nix-channel is broken when using non-standard storepaths, youll need to file a bug on nix itself
<clever> ,paste thomasjm
<clever> ottidmes: your even using the same keysize and type as the comic! :D
<clever> ottidmes: https://xkcd.com/538/
<clever> ottidmes: the universe will die before your key is cracked, it will be faster if we get a rubber hose :P
<clever> thomasjm: what about `type nix-channel ; nix-channel -vvvvvvv --update` ?
<clever> good luck guessing a 2048bit keypair!
<clever> ottidmes: i just configure ssh to never accept passwords
<clever> thomasjm: nix-channel likely hasnt been tested with that, let me see
<clever> thomasjm: did you modify what path /nix is at?
<clever> thomasjm: can you pastebin the whole output from `nix-channel --list ; nix-channel --update` ?
<clever> thomasjm: behind the scenes, nix-channel will use nix-env for you, to manage the channels
<clever> stepcut: oh, there is also `nix ping-store --store ssh-ng://...` to test any given store uri
<clever> ssh-ng://user@host?ssh-key=$HOME/.ssh/id_rsa
<clever> stepcut: oh wait, one sec
<clever> stepcut: you might get away with setting SSH_AUTH_SOCK correctly for the nix-daemon service
<clever> fetchGitPrivate is the only thing that uses NIX_PATH like that
<clever> stepcut: ssh-ng:// just runs ssh directly, and doesnt care about NIX_PATH
<clever> stepcut: yep, now `strace -ff -o logfiles -p <pid>` with the pid of the current nix-daemon, and then run a single build, and ctrl+c the strace
<clever> stepcut: strace needs -f to detect the ssh attempts
<clever> stepcut: and then read the logfiles for where it makes the ssh session
<clever> stepcut: `strace -o logfiles -s 3000 nix....`
<clever> stepcut: try playing with those files until it works
<clever> stepcut: .bashrc vs .bash_profile, its not adding .nix-profile/bin to PATH for non-interactive sessions, a common issue
<clever> stepcut: what happens if you run `ssh user@host nix-store --version` ?
<clever> feep: when on a multi-user install, it will also have a channels_root pointing to roots channels
<clever> lrwxrwxrwx 1 clever users 44 Oct 11 2015 /home/clever/.nix-defexpr/channels_root -> /nix/var/nix/profiles/per-user/root/channels
<clever> feep: and that must have channels within it
<clever> feep: a subdir of it should symlink there
<clever> feep: re-running nix-channel --update or sourcing nix.sh may fix it
<clever> feep: ~/.nix-defexpr/ is to blame
<clever> colemickens: correct, it has some sane defaults for nix.conf, and can function with /etc/nix entirely missing

2018-11-05

<clever> thomasjm: you can also clone nixpkgs, and then nix-env -f ~/nixpkgs -i awscli
<clever> thomasjm: it looks in ~/.nix-defexpr to find nixpkgs by default
<clever> thomasjm: and ~/.nix-defexpr should be fixed the first time you run nix-channel
<clever> thomasjm: ~/.nix-profile will remain broken until you first run `nix-env -i something`
<clever> jabranham: and your still on aarch64?
<clever> jabranham: which channel are you on?
<clever> jabranham: it should, what is the storepath of the kernel its trying to build?
<clever> ddellacosta: `nix-env -e home-manager-path` to uninstall it from .nix-profile
<clever> ddellacosta: then it will be listed somewhere in `nix-env -q`
<clever> ddellacosta: ls -lh ~/.bin
<clever> fede_run: youll need to update the url in the nixpkgs source, and the sha256, then file a PR to nixpkgs
<clever> fede_run: somebody deleted it from the server, so the nix code will need to be fixed
<clever> as well
<clever> ,-A fede_run
<clever> fede_run: you want -i, not -I
<clever> lucky :P
<clever> yeah
<clever> maximiliantagher: in cardano's case, that template haskell was too deep into the dep stack, so it was recompiling 30+ minutes worth of code, when only the top layer has changed
<clever> (nix also deletes .git, so the fallback wont work)
<clever> that older one will read $GITREV, and if it fails, it runs `git rev-parse HEAD`
<clever> maximiliantagher: those just set the env var GITREV at compile time, then you use template-haskell to embed that env var into the code
<clever> maximiliantagher: there are simpler ways to do it, but they result in nix rebuilding the binary every time any commit happens, so you loose caching
<clever> maximiliantagher: 5 pre-allocates a dummy space for a gitrev at compile time, 3/4 then modify the binary (after compiling) to inject a rev, which allows nix to cache the compile haskell code (for when no haskell changes occur), but still update the gitrev within it, so --version remains right
<clever> maximiliantagher: oh, and the cardano example is an even better fit for your answer, one sec
<clever> maximiliantagher: when you name an input foo, and then your release.nix does { foo }:, you get passed an attribute set containing its git sha
<clever> judson: i would prefer it to just silently do the right thing when you have spaces in the field
<clever> ,escape''
<clever> judson: yeah
<clever> fileSystems."/home/clever/VirtualBox\\040VMs" = { fsType = "zfs"; device = "amd/vbox"; };
<clever> judson: ive got a space in one of my mount points, and yes, it emergency-mode'd my machine several times
<clever> oh, he left
<clever> jtojnar: what about the journal when it failed?
<clever> Judson1: probably safer to just use ${pkgs.systemd}/bin/foo
<clever> :(
<clever> Judson1: udev is part of systemd now, you cant win :P
<clever> Judson1: you can also use services.udev.path = [ pkgs.xorg.something ]; to make it available to all rules
<clever> when setting the udev rule from configuration.nix
<clever> Judson1: you could just use the right ${pkgs.xorg.something}/bin/xautolock
<clever> jonreeve: i'm thinking: nix why-depends $(readlink /nix/var/nix/profiles/per-user/jon/profile-233-link) /nix/store/dzkjnv2pw3vaw4rhalr3cl95652knqln-pandoc-2.2.1
<clever> elvishjerricco: yep, thats the next step
<clever> jonreeve: it does
<clever> jonreeve: and which one did --query --roots blame on 231/232/333 ?
<clever> jonreeve: what was the full storepath for pandoc?
<clever> then you installed it in 231, and its still in-use by 233, after 2 other unrelated changes occured
<clever> jonreeve: what 3 did it list?
<clever> jonreeve: what number is ls -lh /nix/var/nix/profiles/per-user/jon/profile
<clever> jonreeve: some of those are due to rollbacks, and you may have already uninstalled them
<clever> to find out how a root connects to a given pandoc, use `nix why-depends`
<clever> jonreeve: if any have no roots, you can `nix-store --delete /nix/store/foo` them
<clever> jonreeve: run `nix-store --query --roots` on each one
<clever> grp: nix and nginx do that i believe
<clever> and its no longer the answer to everything, lol
<clever> and if it fails to parse, the resulting nix.conf will never be saved to /nix/store/
<clever> grp: the same derivation that creates nix.conf (mainly via a bash heredoc) also runs `nix show-config` to parse it
<clever> catern: ive also done: nix-shell -A somepackage, then ran unpackPhase
<clever> exarkun2: a basic override that changes nothing
<clever> > haskell.lib.overrideCabal haskellPackages.yaml (drv: {})
<clever> exarkun2: which one you want, depends on what your trying to modify
<clever> exarkun2: overrideCabal applies before generic-builder.nix
<clever> exarkun2: it works, but it applies after generic-builder.nix and they cease to be "haskell packages"
<clever> exarkun2: yes and no
<clever> ghusbands: you can also use things like `nix copy` to create a binary cache with all of the build products, as either an s3 bucket, static http server, or just a tarball
<clever> ghusbands: if you import a specific version of nixpkgs, an store all the expressions in git, you can reproduce the whole thing, as long as the src URL's arent deleted
<clever> heh, hit the magic number!
<clever> das_j: probably not
<clever> das_j: that handled by setup hooks usually, which only get ran by nix-shell and nix-build
<clever> eeva: whatever the problem is, you already fixed it, but nix-daemon had cached the bad base64
<clever> eeva: what happens if you `sudo systemctl restart nix-daemon.service` then try `nix-store -r` again on a storepath?
<clever> which is a unix socket to nix-daemon
<clever> connect(3, {sa_family=AF_UNIX, sun_path="/nix/var/nix/daemon-socket/socket"}, 110) = 0
<clever> eeva: it read that error over fd#3
<clever> "\0\0\0\0\0\0\0invalid character in Base64 string\0\0\0\0\0\0\1\0\0\0\0\0\0\0", 32768) = 432
<clever> eeva: and then throw all og the logfiles into a pastebin, gist is better for multiple files
<clever> eeva: all thats left is trace, strace -ff -o logfiles -s 3000 nix-store -r /nix/store/gkwf9j3k83jbgilg38j624l0azkjna61-mirrors-list
<clever> eeva: yeah, that seems like every single place a base64 could come from
<clever> eeva: and the above curl command output what?
<clever> eeva: oops, ^^
<clever> exarkun2: it should just download that from the binary cache
<clever> exarkun2: yeah, thats how things should work
<clever> eeva: can you pastebin `nix show-config` ?
<clever> eeva: yeah
<clever> eeva: can you pastebin `nix show-config` ?
<clever> -r
<clever> eeva: and what about running nix-store on each of those paths?
<clever> eeva: and what is the contents of ~/.config/nix/nix.conf ?
<clever> eeva: try `nix-store -r` on each path listed in your last pastebin, what ones work and what dont?
<clever> eeva: and if you run `nix-store -r /nix/store/dwiqyq7z55pml2zb20shs88xbiimx6q7-source.drv` does it work?
<clever> eeva: can you pastebin that .drv file?
<clever> building of '/nix/store/dwiqyq7z55pml2zb20shs88xbiimx6q7-source.drv': woken up
<clever> eeva: try the -vvvvv again, but also add -j 1, then pastebin that log
<clever> eeva: can you pastebin your nix.conf file?
<clever> eeva: what command did you run to cause this?
<clever> adamt: i believe they all just set a prio or 1500, and then it will be the order of the modules being imported
<clever> adamt: what about mkAfter ?
<clever> betaboon: yeah
<clever> betaboon: the attribute set the function was called on, is stored in args
<clever> thomasjm: not really

2018-11-04

<clever> aanderse: yeah
<clever> with support for the steam overlay
<clever> its just a fork of wine, baked into steam
<clever> tilpner: that can happen if you ctrl+c a previous gc, or had a lot of failing builds
<clever> steveeJ: hostname i dont think is changed
<clever> steveeJ: i think it does, but the network is online at the time
<clever> if you dont reboot for 3 months
<clever> steveeJ: the more fun part, is that it works at nixos-rebuild, so you leave it for 3 months, and then GC all old generations :P
<clever> then it tries to continue the boot, and fails
<clever> causing systemd to never wind up in PATH
<clever> with nixos, any error in the activation scripts results in it partially aborting the boot
<clever> ah, home-manager is safer then
<clever> steveeJ: that will leave your maching unable to boot
<clever> steveeJ: never attempt to do network in an activation script
<clever> steveeJ: the deep internals of nix
<clever> steveeJ: outputHashMode = "recursive"; allows it to instead be a dir
<clever> steveeJ: outputHashMode = "flat"; is what required $out to be a file, not runCommand
<clever> steveeJ: why do you need .git?
<clever> steveeJ: if your build fails to meet the defined outputHash, then nix will consider it a failure
<clever> steveeJ: runCommand "name" { outputHashAlgo = "sha256"; outputHash = "...."; outputHashMode = "flat"; } "script"
<clever> steveeJ: one sec
<clever> lukego: if it was booting via legacy, it would be more likely to have a proper menu
<clever> lukego: systemd-boot is efi only
<clever> lukego: i prefer using grub, and ive heard that systemd-boot lacks the rollback menu
<clever> lukego: after you boot from something else, you can use `nixos-enter` to chroot into it (assuming you mounted everything right), and can then `nixos-rebuild boot`
<clever> catern: i believe the product of builtins.toFile must have zero dependencies, you want pkgs.writeText or pkgs.writeTextFile
<clever> symphorien: try patchShebangs instead, since it knows what a #! looks like
<clever> symphorien: i then spent 2 hours helping them debug why the zip file was corrupt :P
<clever> symphorien: i have seen somebody run a command like that before, which replaced paths inside a zip file
<clever> and it sounds like the postInstall code in cabal2nix expects it to always be on
<clever> yep
<clever> thefloweringash: enableSeparateDataOutput = true; has to be set on the derivation, which cabal2nix should be doing
<clever> thefloweringash: mostly nix
<clever> thefloweringash: $data will only exist if a special bool is set when running mkDerivation
<clever> nschoe: the version of state like db.sqlite
<clever> ah, then it shouldnt have too much trouble with the CFG leaks in windows
<clever> angerman: do you happen to know if the RTS generates any chunks of code at runtime, such as with thunks, requiring pages to be mapped executable?
<clever> ah
<clever> catern: and using those vars, you can see if a function is being called too much
<clever> ,profiling
<clever> (with the same args)
<clever> catern: but there is no cache for function calls, so you can wind up wasting a lot of cpu and ram by re-calling the same function repeatedly
<clever> catern: there is an import cache that maps paths to values, so it should cache the result of importing
<clever> not sure then
<clever> ottidmes: is this on nixos?
<clever> ottidmes: did you edit the file in the store?
<clever> ottidmes: the fork from my example has had some improvements to fix various things, not sure if they got int nixpkgs or not
<clever> ottidmes: are you using the fork of yarn2nix from the example i linked?
<clever> that should also work i believe
<clever> probably with `yarn install`
<clever> ottidmes: you need to generate one outside of a nix sandbox, and commit it to your repo

2018-11-03

<clever> joepie91: if you uninstall pkgconfig, then it will instead say pkgconfig not found, and you will then add it, and all is well
<clever> joepie91: and to make things worse, when you do forget to add it to your nix-shell inputs, it claims freetype2 not found, when it exists
<clever> if you wrongly install it with nix-env/systemPackages, then it just never finds buildInputs
<clever> that only works under nix-shell and nix-build
<clever> so it magically finds all inputs
<clever> pkgconfig has a setuphook, that will scan all buildInputs, and add them to the pkgconfig search path
<clever> joepie91: it doesnt work right, and only leads to more confusion when it half works due to not being in the nix-shell
<clever> joepie91: also, if you ever installed pkgconfig in nix-env or systemPackages, remove it asap
<clever> joepie91: and pkgconfig in the nativeBuildInputs?
<clever> Zajcev_: investigate how to configure it to look elsewhere for pam modules, and then fix either the package or nixos module
<clever> joepie91: cmake is in cmake
<clever> joepie91: cmake!
<clever> Zajcev_: its over in pam_pgsql, sounds like a bug in vsftpd's package
<clever> ,locate pam_pgsql.so
<clever> joepie91: just add pkgconfig and alsaLib to your inputs, and it should work
<clever> ddellacosta: the only other option is to mess with nix-copy and a usb stick, which is more complex
<clever> ddellacosta: nixos-install is just a helper that runs nixos-rebuild under a chroot for you
<clever> ddellacosta: boot the install media again, mount the existing fs to /mnt (and /mnt/boot if you have it), edit the config, and nixos-install
<clever> ddellacosta: got an ethernet cord?
<clever> while keeping the same API
<clever> ottidmes: if you pass fetchSubmodules=true; then it will internally switch to pkgs.fetchgit for you
<clever> ottidmes: fetchFromGithub "supports" submodules now
<clever> fresheyeball: not sure, he has a PR open with a large number of improvements
<clever> ottidmes: cant find it on nixpkgs, but i can pull up the other example i have
<clever> fresheyeball: he has been fixing some ghc problems lately, including things like commandline too long
<clever> fresheyeball: some of angerman's recent changes might fix it, but i'm not sure what branch they are on
<clever> fresheyeball: thats over 50, lol
<clever> ottidmes: yarn2nix is what you want, i think the recent terraform PR's have an example
<clever> fresheyeball: how many deps does this package have?
<clever> fresheyeball: what command is generating a cmdline too long?
<clever> stepcut: and you dont even really need to run cabal2nix, because of callCabal2nix: https://github.com/input-output-hk/cardano-chain/blob/devops-1118/pkgs.nix#L11-L15
<clever> stepcut: its a 2 lines, the example --shell makes is far more complex then it has to be
<clever> skip --shell and just write a better file
<clever> just load it with haskellPackages.callPackage
<clever> you can ignore the default.nix cabal2nix generates, and only use the main file it produces
<clever> just do the right thing from shell.nix to begin with, dont try to do magic things that guess what you want
<clever> stepcut: i try to avoid using inNixShell, it just makes things behave in unpredictable ways
<clever> yeah, it sounds like its just poorly written
<clever> nschoe: if you found that inside a .img, then it is built, but not very well
<clever> nschoe: you will need the right entries under boot.loader and fileSystems."/" and "/boot" to make sure it still boots, those will depend on how exactly the image is built
<clever> WilliamHamilton[: if emacs is searching the right areas of ~/.nix-profile/, and the package had those files in its $out, yes
<clever> nschoe: ah, then the default config is likely wrong
<clever> nschoe: that will merge everything defined in that file with the other things you add to configuration.nix
<clever> WilliamHamilton[: but you can do something like `nix-env -f default.nix -iA foobar` to install something into your profile, it will get merged into ~/.nix-profile/
<clever> WilliamHamilton[: nix-build doesnt have -i or -p flags