2017-08-07

<clever> nwuensche: and what command fails with an error?
<clever> nwuensche: what does "ping cache.nixos.org" say?
<clever> nwuensche: but if curl fails the download, xz gets a 0 byte input
<clever> nwuensche: nix will pipe curl directly into xz to uncompress the nar.xz's
<clever> nwuensche: and what is in /etc/resolv.conf
<clever> for example, pkgs.ncdu just becomes /nix/store/<hash>-ncdu-1.2.3
<clever> derivations turn into directories automatically
<clever> yeah
<clever> joehh: if <boolean> then pkgs.fetchgit { ... } else ./localdir
<clever> nix-env keeps track of every version you install, and wont let you GC things
<clever> joehh: something else, if your only testing, try to stick to nix-build, rather then nix-env
<clever> though the name will be foobar, rather then foo
<clever> spacefrogg: you dont even need that, just ./foobar is enough to copy that dir
<clever> MoreTea: there is also runCommand
<clever> mg: you may want to look into nix-shell --run
<clever> mg: nix-shell spawns a new shell, and only returns once the user exits from that
<clever> while configurePhase replaces the entire default
<clever> dev1: preConfigure runs before the default configurePhase
<clever> LinArcX: what kind of thing do you want to debug?
<clever> LinArcX: debug it how?
<clever> joehh: i'm more of a nightowl, so i can see how we might be on at the same times
<clever> lol
<clever> NB canada, atlantic standard time
<clever> joehh: nix-env -r
<clever> and it wont accept unknown argumentsd
<clever> because your module may break in 6 months when anybody adds a new argument to all modules
<clever> nixos may pass it more, but only config and pkgs get used
<clever> config and pkgs are the only ones it will use
<clever> it ignores all OTHERS
<clever> yep
<clever> it only works in the function arguments like that
<clever> dev1: that lets it ignore any other arguments it was given
<clever> Dezgeg: armv7 builds seem broken
<clever> /nix/store/c57l8jl8g9b3ind5ihd3jy3m5g4n0q37-binutils-2.29/bin/ld: unrecognised emulation mode: armelf_linux_eabi
<clever> Dezgeg: ping?
<clever> and nix will directly execute ghc, and drop an ELF in /nix/store/
<clever> you can also just do builder = "${ghc}/bin/ghc"; args = [ "-o" (builtins.placeholder "out" ./input.hs ]; name = "binary"
<clever> but with a few tweaks, it could run entirely without the stdenv
<clever> NickHu: in this example derivation i made, it runs the bash in stdenv.shell against a builder.sh i inlined
<clever> how you create $out and what it does, nix doesnt care
<clever> NickHu: at the lowest level, all nix does is run a set builder, with the given args, and pass it a $out env variable
<clever> ive never messed with auth over nfs, and i avoid uid mappings as well, i just make sure the uid's match between all machines

2017-08-06

<clever> NickHu: but 99% of the time, you already have glibc in your nix store, so your not really saving space by trying to exclude it
<clever> which it used to unpack and patchelf the bootstrap tools
<clever> i can only think of one area in nixpkgs that absolutely requires a static binary, it has a staticly linked busybox, not even in a tar, grabbed by call to <nix/fetchurl.nix>
<clever> thats about the only benefit it has
<clever> in theory, you can just tar up a static ELF and ship it to any distro, without having to drag nix along
<clever> yeah
<clever> thats uses by nixos to create the initrd images
<clever> so you have the dynamic libs, but none of the other junk (locales, docs, share files ...)
<clever> NickHu: this copies the ELF programs and the .so files into a new derivation, then patchelf's the rpath to fix it
<clever> ah
<clever> but you may also run into other libraries that lack static files
<clever> that is in the qemu expression i linked
<clever> NickHu: or just use dynamic, nix solves nearly all of the typical reasons you would want static for
<clever> NickHu: you need to either find or create static libraries for everything: https://github.com/cleverca22/nix-misc/blob/master/qemu-user.nix
<clever> avn: last i looked at it, the policies are just javascript files running in spidermonkey, that take in some state and return true/false
<clever> avn: thats more in the area of not-os, where i used runit
<clever> as long as the host has mkdir, mount, touch, mknod, and chroot in $PATH, it can enter the container
<clever> the goal was to be able to launch a systemd-nspawn like container, without systemd on the host
<clever> this is a related thing i worked on
<clever> so it will more happily run in a container, with full systemd
<clever> that will remove the kernel component and re-configure a few things
<clever> oh, and something else you can then try, boot.isContainer = true; and build nixos
<clever> but you probably want a chroot anyways
<clever> dtzWill: yeah, once you switch over to a nix built by nix, it may get upset at the lack of /etc/
<clever> avn: yeah
<clever> 98% of the time, your going to replace it with a nix-built version anyways
<clever> dtzWill: its sort of like building a /usr/bin/nix-build for ubuntu
<clever> dtzWill: i'm also not quite sure you even need the android ndk
<clever> avn: taktoa has been working on a project that turns a ninja file into a dervation with one .o per dervation
<clever> that basicaly re-implements ccache using the nix store
<clever> avn: incremental builds, one derivation per .o file
<clever> the file i pasted above solves that
<clever> so you have to -f '<nixkgs>' every time
<clever> avn: nix-env basicaly ignores the search path
<clever> because of this file, i can now do nix-env-iA foo.wireshark-gtk, and it grabs it from master
<clever> import /home/clever/apps/nixpkgs
<clever> [clever@amd-nixos:~]$ cat .nix-defexpr/test/foo/default.nix
<clever> i also recently read the code behind .nix-defexpr, and can now do fun things with it
<clever> avn: the entire nixpkgs set contains a pkgs attribute, that points to itself
<clever> avn: for a git checkout, you just want nix-env -f /home/clever/apps/nixpkgs -A wireshark-gtk
<clever> the name before the . comes from the names in "nix-channel --list"
<clever> tommd: then you want nix-env -iA nixpkgs.wireshark-gtk
<clever> tommd: are you on nixos or another distro?
<clever> tommd: nix-env -iA nixos.wireshark-gtk
<clever> LinArcX: the type is string, so you need to either embed it right into configuration.nix as a string, or use builtins.readFile
<clever> LinArcX: https://nixos.org/nixos/options.html#services.tor.extraconfig and also .enable
<clever> avn: Xephyr demands a valid xkb dir on the command-line
<clever> ive also had to deal with this when running xnest, xephyr, and xvnc
<clever> [clever@amd-nixos:~/apps/nixpkgs]$ ls /nix/store/5nidmghgw700wkj4pycg9a80q25qbxzb-xkeyboard-config-2.21/share/X11/xkb/
<clever> /nix/store/5nidmghgw700wkj4pycg9a80q25qbxzb-xkeyboard-config-2.21
<clever> [clever@amd-nixos:~/apps/nixpkgs]$ nix-store -qR /run/current-system | grep keyboard
<clever> Infinisil: in the xkb package
<clever> i recently noticed that the giant help button opens the help in vim
<clever> esc is also in the right spot
<clever> and the super/alt are also swapped
<clever> ive got a keyboard like this, but control/capslock are in the "right" place: http://xahlee.info/kbd/i/kb/sun_keyboard_left.jpg
<clever> so the program must wait and see if there is more
<clever> LnL: part of the problem with escape in a terminal, is that things like the arrow keys send escape and another key
<clever> Profpatsch: nn
<clever> yeah, stackoverflow stopped that from happening
<clever> LnL: it was some weird combo of bash, tar, and something custom, and the ELF inside the tar handles the custom part
<clever> LnL: while trying to unpack some vmware bundles, i was using replace mode to change it from running the unpatchef elf to just unpacking, and i discovered vim appending a \n to the end of the file, breaking it
<clever> Profpatsch: did you see my test-ghc.nix expression?
<clever> Profpatsch: testing a build with the new expression...
<clever> aka, $out//share/x86_64-linux-ghc-8.0.2/shake-0.15.11/html
<clever> and if i disable all split-output, the docs land here
<clever> [clever@amd-nixos:~/apps/nixpkgs]$ ls -lh /nix/store/sp11wb69bhy00g7wqhls4md9mdi1qzdm-shake-0.15.11/share/x86_64-linux-ghc-8.0.2/shake-0.15.11/html/
<clever> ahh
<clever> this variant does build
<clever> nix-repl> :b haskell.lib.overrideCabal haskellPackages.shake (drv: { enableSeparateDocOutput = false; enableSeparateDataOutput = false; })
<clever> then the docs would land directly in $out/html
<clever> ah yeah, i tried to get rid of the double ghc in that path, and thats when i discovered the instability
<clever> and this part explains the $data/share/ghc-8.0.2 prefix
<clever> (optionalString enableSeparateDataOutput "--datadir=$data/share/${ghc.name}")
<clever> and putting it in data causes a cycle
<clever> because the default when absent isnt stable
<clever> i'm thinking, we must always provide --docdir
<clever> heh
<clever> and data refers to out
<clever> and $out refers to data
<clever> the docs land in $data/share/ghc-8.0.2/doc/x86_64-linux-ghc-8.0.2/shake-0.15.11/html/src/Paths_shake.html
<clever> also, enableSeparateDocOutput = false breaks the build
<clever> the castle of purity is built upon a mountain of impurities
<clever> yes
<clever> but --data-dir=/foobar/ wont contain the docs
<clever> the crazy part, is that --data-dir=/fooghcbar/ will contain the docs
<clever> also setting --doc-dir and --html-dir prevents that problem
<clever> but if ghc is not present, the docs land in --prefix=
<clever> if the --data-dir= contains the string "ghc", the docs go into it
<clever> oh, this reminds me, there is a bug in cabal itself
<clever> nix-repl> :b haskell.lib.overrideCabal haskellPackages.shake (drv: { enableSeparateDocOutput = false; })
<clever> so the question becomes, where do the docs land if we dont split
<clever> "/nix/store/18w3lpr7dvwgid5m6la2jb4n744gx2dv-shake-0.15.11-doc/share/doc/html"
<clever> nix-repl> (haskellPackages.shake.doc or haskellPackages.shake.out) + "/share/doc/html"
<clever> if a given attr is missing, use the 2nd value
<clever> nix-repl> haskellPackages.shake.docs or "not there"
<clever> "not there"
<clever> Profpatsch: i also wonder if this conf file would fix your problems?, it always points to wherever cabal was told to put docs
<clever> so $doc must never refer back to $out
<clever> Profpatsch: a .conf file within $out has the paths to haddock files in $doc
<clever> lib/ghc-8.0.2/package.conf.d/shake-0.15.11-JGGEmqTyo9u3rsNdYtylaz.conf:haddock-html: /nix/store/18w3lpr7dvwgid5m6la2jb4n744gx2dv-shake-0.15.11-doc/share/doc/html
<clever> lib/ghc-8.0.2/package.conf.d/shake-0.15.11-JGGEmqTyo9u3rsNdYtylaz.conf:haddock-interfaces: /nix/store/18w3lpr7dvwgid5m6la2jb4n744gx2dv-shake-0.15.11-doc/share/doc/html/shake.haddock
<clever> Profpatsch: one minute...
<clever> and rather then force them to debug/fix them entirely, you can just enableSeparateDocOutput = false; to merge the docs into another output
<clever> Profpatsch: yeah, but new packages may introduce new cycles in different files
<clever> i had to nuke all references to $out in the html to get rid of the cycles
<clever> 335 remove-references-to -t $out $x
<clever> 334 for x in $doc/share/doc/html/src/*.html; do
<clever> the Paths_ module in cabal already causes that problem
<clever> doHaddock = true; enableSeparateDocOutput = false; would solve that
<clever> and nix will fail the build hard
<clever> and $out has a haddock path in it
<clever> down the road, somebody may create docs that refer to $out
<clever> one reason i can think of to keep it, is to fix cycle problems
<clever> ah, i see now, enableSeparateDocOutput isnt used currently and defaults to doHaddock
<clever> dont remember right now
<clever> to set it correctly for all of haskellPackages
<clever> peti had made changes to cabal2nix, that scan the cabal file, and figure out if it has docs or not
<clever> nix staying that foo.doc doesnt exist will at least tell you, it probably doesnt have docs
<clever> was thinking it might lead to confusion, like you grabbed foo.doc, and then you dont have any docs???
<clever> so if you say you have a doc output, you must make $doc
<clever> the problem is that nix will fail the entire build if you dont create $doc
<clever> just an empty directory
<clever> if the package doesnt have any docs, it will result in an empty $doc output
<clever> Profpatsch: i believe if you turn off split docs, they just land in $out or $data
<clever> ah, i wasnt 100% sure on where the docs had to be within $doc
<clever> Profpatsch: how far back?
<clever> process tree and backtraces in the gist
<clever> the make process locked up, and hasnt been doing anything for days
<clever> i discovered it, because hydra says i have been building linux for 2 days 12 hours
<clever> ive also got another datapoint on things (git and make) randomly deadlocking on nixos: https://gist.github.com/cleverca22/b8c04ac38a05660539a9b2157a9fa0d7#file-make-txt
<clever> i'm cheating a bit by using nix-store -r to create roots
<clever> i think nix-instantiate has its own flag to make roots
<clever> Infinisil: and if pointed to an already built storepath, it only protects that and its runtime closure
<clever> Infinisil: if you point it to a .drv, it will build it, and protect the default output only
<clever> Infinisil: nix-store -r will try to build whatever you point it to
<clever> DavidEGrayson: then make a root for that buildEnv derivation
<clever> DavidEGrayson: ah, one option is to create a new derivation, that just passes all 3 gcc's to buildEnv
<clever> DavidEGrayson: i think just one link to the derivation at the root is enough
<clever> thats what nix-build uses behind the scenes
<clever> yep
<clever> hmmmm, and nixops did nothing to root its own builds, so i have another 400mb to dl
<clever> DavidEGrayson: manualy run this on things you know take 40 minutes, to create a root on them
<clever> DavidEGrayson: the only other trick, nix-store -r --add-root foo.root --indirect /nix/store/foobar
<clever> ive only found that hydra can root things to that level
<clever> LnL: but will that root the build-time deps of everything in the build-time closure?
<clever> g:ycm_seed_identifiers_with_syntax looks handy
<clever> every now and then it tells me i need to make a config file somewhere in my home dir
<clever> i also need to get around to doing that
<clever> but the ycm may have enabled the built-in autocomplete that had previously been disabled
<clever> it never did that until i turned on ycm
<clever> Infinisil: i never got around to properly configuring ycm, ive found it can only complete words already present in the open files
<clever> that might explain the high cpu fan speeds
<clever> temp1: +74.2°C (high = +70.0°C)
<clever> grantwu: heh
<clever> and because it segfaults, git just aborts the ammend
<clever> it happens the most when i "git commit --amend" and dont want to change the commit msg
<clever> LnL: ive noticed a weird problem with ycmd in vim, if i quit vim too quickly, while ycmd is starting, vim segfaults
<clever> leading to .git containing more history
<clever> so you have to fetch a branch that contains that rev
<clever> the git server will also refuse to let you fetch a given revision directly
<clever> git will keep track of branches you didnt ask for, so the latest rev of those can change
<clever> yep
<clever> simpson: ive also heard that the .git folder isnt very deterministic, so the hash will just randomly break
<clever> it was also only added recently
<clever> grantwu: just related to -i vs -iA
<clever> grantwu: its another silly mistake that new users have done repeatedly
<clever> grantwu: and since you didnt tell nix-build what to build, it builds everything in the set
<clever> grantwu: they omited the -A argument
<clever> then throws out every broken package in the entire set
<clever> and nix-build happily tries to build everything
<clever> grantwu: at least twice, ive seen people run nix-build in the root of something like nixpkgs
<clever> NickHu: line 7-13 also explains how the repo is made
<clever> NickHu: so you want to check both the git history on this file, the maintainers, and possibly open an issue on nixpkgs and/or debian.org
<clever> NickHu: if i eval firmwareLinuxNonfree.meta.position in nix-repl, i get pkgs/os-specific/linux/firmware/firmware-linux-nonfree/default.nix:28"
<clever> robot: turning it on will auto-configure everything, and install libvirt
<clever> This option enables libvirtd, a daemon that manages virtual machines. Users in the "libvirtd" group can interact with the daemon (e.g. to start or stop VMs) using the virsh command line tool, among others.
<clever> robot: did you also virtualisation.libvirtd.enable ?
<clever> though it defaults to true
<clever> robot: virtualisation.libvirtd.enableKVM maybe
<clever> qemu works fine, when ran directly
<clever> ah, you didnt mention that
<clever> yay
<clever> robot: it just works out of the box
<clever> and :r to reload every time you change the config files
<clever> maybe :b config.hardware.firmware
<clever> NickHu: yeah, nix-repl '<nixos/nixos>' and then "${config.hardware.firmware}"
<clever> you will want to check in that resulting path, both to confirm it symlinked the right version, and that the files are in the right area
<clever> NickHu: and if you read /etc/modprobe.d/nixos.conf, you will find the path to the firmware, after nixos has merged foo and non-free
<clever> so you have clear control of the order
<clever> so its best to hardware.enableAllFirmware = false; and then hardware.firmware = [ foo pkgs.firmwareLinuxNonfree ];
<clever> but i'm not sure where all-firmware.nix will put it
<clever> the order of entries in hardware.firmware will control that then
<clever> ah
<clever> NickHu: and removing the enableAllFirmware bit will prevent the conflict
<clever> 2017-08-06 01:01:09 < clever> if you remove the scan file, you can just do something like this
<clever> you will need to adjust the mkdir and cp to recreate the right layout
<clever> the cp command in here will adjust copy it from one area to another
<clever> let foo = runCommand "bar" { src = pkgs.fetchFromGitHub { ... }; } "mkdir -pv $out/lib/firmware; cp $src $out/lib/firmware/output" in ...
<clever> the runCommand above creates an entirely new firmware package
<clever> you will either want to add a pkgs.fetchgit, pkgs.fetchFromGitHub, or just clone it by hand and refer to it via ./input-dir
<clever> yeah
<clever> hardware.firmware = [ foo ];
<clever> let foo = runCommand "bar" {} "mkdir -pv $out/lib/firmware; cp ${./input-dir} $out/lib/firmware/output" in ...
<clever> if you remove the scan file, you can just do something like this
<clever> core.c: scnprintf(filename, sizeof(filename), "cal-%s-%s.bin",
<clever> each slot in the machine will load a different firmware file
<clever> which is very weird
<clever> it also has the pci bus-id in the name
<clever> i'm guessing thats calibration data
<clever> yeah
<clever> and if i cross-reference the firmware package and your dmesg, i can see its loading from this region
<clever> $ ls -lh /nix/store/vkf6qyph8sc433sz714bzdhwhi827x3p-firmware-linux-nonfree-2017-03-11/lib/firmware/ath10k/QCA6174/
<clever> so the file "scan/not-detected.nix" doesnt scan for anything, it just blindly enables everything
<clever> the literal "or" on 46 is another thing
<clever> the || on 36
<clever> line 36 is has an or in it
<clever> NickHu: because #1 is in your imports, everything under #2 is active
<clever> though that naming is a bit off
<clever> NickHu: given what i read recently about broadcom chips, and this error, i suspect it might not be finding any firmware, and then defaulting to the firmware in the chip's rom
<clever> [ 7.071837] ath10k_pci 0000:01:00.0: Direct firmware load for ath10k/pre-cal-pci-0000:01:00.0.bin failed with error -2
<clever> NickHu: what is in your configuration.nix and hardware-configuration.nix?, anything named detect?
<clever> from https://nixos.org/nixos/options.html#hardware.firmware i click "declared in" and search for that, no default in this file, but you may have other things
<clever> NickHu: and also you need to create a derivation that puts files at $out/lib/firmware, and then push that derivation into hardware.firmware
<clever> NickHu: ^^^
<clever> oops, did it again, lol
<clever> Nobabs27: dmesg | grep ath10k_pci -C5
<clever> Nobabs27: re-run that grep with -C5
<clever> can you pastebin the output of dmesg?
<clever> dmesg should say what file is using and give more info
<clever> what does dmesg say and what is not working right?
<clever> the driver sends a special message down to udev, saying which file it wants, and udev must then supply that file
<clever> it will always load the one the driver asks for, so you cant easily make it load a different file
<clever> oops
<clever> ive now been on xterm for over a decade, and i cant use any other terminal
<clever> back when i started linux, gnome-terminal was horid slow (my cpu was only ~600mhz), so i used xterm
<clever> yeah, lol
<clever> left and middle click also have their own menus
<clever> yeah
<clever> adelbertc: also, there are some hidden menus in xterm, hold control and hold right click