2017-03-04

<clever> another more painful part, is that the generated nix file depends on the source tar
<clever> but you would need to re-implement the makefile in nix, by hand (or write a generator)
<clever> in theory, it could do that
<clever> so you get both distcc and ccache, in a pure manner
<clever> and it can use build slaves to split something like chromium or linux over 200 slaves
<clever> so it can reuse a .o between builds, in a pure manner
<clever> and with the filterSource stuff i did, the .o file hash depends purely on the hash of the source (and cli args)
<clever> my idea, was to make every single .o file its own derivation, that only depends on the .h files it references
<clever> spacekitteh: this is something similar i threw together: https://github.com/cleverca22/nix-tests/blob/master/make.nix
<clever> gchristensen: is that why i saw a "mast" branch a few days ago?
<clever> cant think of anything else then
<clever> devoid: anything in the dropdown?
<clever> devoid: try draging that monitor?
<clever> devoid: and df in general, the size of every mount-point is in constant flux
<clever> devoid: cacti/snmp is also confused, it refuses to let me graph a 2nd filesystem within the same pool
<clever> spacekitteh: yes
<clever> but it needs xfce in full control to persist, so it can restore upon login
<clever> i believe it uses xrandr to apply the changes, so you can probably just run it under nix-shell to test
<clever> ah, i was wondering if xfce4-display-settings helped any
<clever> devoid: are you using xfce?
<clever> my driver default to mirror
<clever> i think its more to do with the xorg default, span or mirror
<clever> devoid: ow, lol
<clever> but the option definitions themselves arent protected
<clever> mkOption sort of adds a type system to the options framework
<clever> was that the pkgs default.nix?
<clever> no idea why it was setup like that
<clever> devoid: by adding other forks of nixpkgs like that, i can checkout branches from any of them, and switch back to master easily, so i only need 1 copy of the nixpkgs git repo
<clever> devoid: if you want to develop or edit expressions in a channel line 6/7 of http://pastebin.com/r5pDRFhv
<clever> devoid: if you just want to use a channel, what gchristensen said
<clever> but i need to finish this code
<clever> i should get sleep soon, lol
<clever> what gchristensen said
<clever> nixpkgs-channels
<clever> oops
<clever> devoid: also, the release-17.03 branch on nixpkgs isnt the safe one, you want release-17.03 of nixpkgs-master
<clever> devoid: the last link i gave is the 17.03 version of nvidia's package
<clever> devoid: you can change branches right on the github web-ui
<clever> 16.09 has a seperate beta.nix rather then an all-in-one
<clever> devoid: the branches on this repo corespond to nix channels, so you can just peek at any channel and find the answer
<clever> heh
<clever> ah
<clever> the pi was off?
<clever> ive also had systemd refuse to shutdown because nfs is busy umounting, when there is less then 30 seconds left on the UPS, and the entire network is down because of a power outage!
<clever> ive had issues before where a normal umount also hangs, because the nfs client is dumb and wants a server up to disconnect cleanly
<clever> "umount -l" is a lazy umount, it will un-hook it from the visible FS's and let it take its time (potentialy forever) in the background where it wont bother anything
<clever> Acou_Bass: its more likely that the nfs server is to blame
<clever> Acou_Bass: umount and the problem should go away, if umount hangs, there is umount -l
<clever> Acou_Bass: up next is "strace df -h" to find out which mount it is
<clever> Acou_Bass: ah, so stdin/stderr/stdout are all pointing to /dev/pts/3, and it has no open files, but its probably doing stat on an nfs mount
<clever> df -h is usualy enough to check the nfs mounts
<clever> Acou_Bass: and are any nfs mounts hung?
<clever> Acou_Bass: it appears to be blocking on disk io, what does ls -l /proc/2165/fd/ say?
<clever> Acou_Bass: can you pastebin "ps -eH x"'s output when its hung?

2017-03-03

<clever> magnetophon: but you can just download any ISO, and switch the channel to 17.03 after it boots
<clever> magnetophon: oh, and i dont think 17.03 has actualy released officialy yet
<clever> magnetophon: only other thing i can think of is strace, which will generate several MB worth of logs, and include IP's that you appear to be censoring
<clever> so you can use nixos-install to upgrade any machine that isnt capable of upgrading itself
<clever> nixos-install is basicaly just a script for running nixos-rebuild under a chroot
<clever> one solution is to just boot it from an install ISO, add the new channel as nixos, mount the original filesystems up, and then re-run nixos-install
<clever> do you have physical access to the problem machine?
<clever> and you dont need to use root on the remote machine, only the local machine
<clever> magnetophon: nix-copy-closure --from root@2.2.2.2 /nix/store/vjc48rp9aq3cqikdrsl8pradnq9ndrif-nixos-rebuild
<clever> magnetophon: and if you run that as root, it will bypass the signing checks for things it pulls in
<clever> magnetophon: you can also use nix-copy-closure --from to pull things in, rather then push
<clever> magnetophon: try removing the custom 2.2.2.2 cache from the config and see if the problem goes away
<clever> magnetophon: lines 17-20 look like you used " rather then '', and now the file just looks weird
<clever> magnetophon: and the output of nix.conf
<clever> can you pastebin the output of env?
<clever> and no use of --option binary-cache-public-keys ?
<clever> magnetophon: it contained this line after you finished editing?
<clever> binary-cache-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
<clever> magnetophon: next time nixos-rebuild does (test|switch), it will restore nix.conf back to the symlink it should be
<clever> magnetophon: try this, "cd /etc/nix; cat nix.conf > nix.conf.copy; rm nix.conf; mv nix.conf.copy nix.conf" then just edit nix.conf and put in the right config
<clever> the -1 is part of the key name, but it is NOT part of the url
<clever> magnetophon: "--option binary-caches cache.nixos.org-1" is an invalid url, and you must use the correct pubkey for it to work against the real cache.nixos.org
<clever> magnetophon: i mean the binary cache key
<clever> magnetophon: how did you try adding the public key for signing?
<clever> magnetophon: and does root have a pw set?, check /etc/shadow
<clever> magnetophon: get a root shell via sudo and double-check the ssh keys are in place correctly
<clever> magnetophon: either the ip has changed, or somebody is performing a mitm attack and now has your pw :P
<clever> magnetophon: you need to either use --to root@other-machine, setup signing, or make the target user a trustedUser
<clever> MichaelRaskin: my screens are laid out as 1,3,2, so that wouldnt look right
<clever> devoid: the defaults on my system is to mirror it 3 ways
<clever> devoid: only downside, is that those settings only take place after login, but i dont really need 3 monitors working correctly to type in a name+pw, lol
<clever> devoid: i just drag them around in the xfce display settings
<clever> bobthejanitor: yeah, and put in any info you can gather on the issue
<clever> bobthejanitor: not sure then, i usualy see this kind of problem if somebody ran nixos-rebuild when /boot wasnt mounted, then the files wind up in the /boot folder of /, rather then the /boot partition
<clever> bobthejanitor: and is /boot correctly mounted?, does it show up in df -h or mount?
<clever> bobthejanitor: does that file exist in /boot/ ?
<clever> dmj`: nixos-unstable is also on nix version 1.11.6
<clever> or put the version into the name, so it always mismatches
<clever> tnks: you need to change the hash, to make it re-download, and tell you the correct hash
<clever> so it will match the old file you previously downloaded, and not update
<clever> tnks: and you bump it to version 1.3, the name in the /nix/store wont change
<clever> tnks: if you have a fixed-output download of http://example.com/1.2/project.tar.gz, with a given sha256 in the fetchurl
<clever> though github does accept a name/pw pair over basic auth
<clever> netrc partialy handles that, but it only has domain level accuracy i believe
<clever> dmj`: nix doesnt really use https much for fixed-output stuff, because any mitm is caught by nix verifying the output hash
<clever> dmj`: or if some of those are not over https
<clever> tnks: so now xorg and glibc servers are getting a copy of your token
<clever> tnks: and also, NIX_CURL_FLAGS will hand your github token out to EVERY tarball server, like its free cookies
<clever> tnks: then .bashrc can setup all the tokens
<clever> tnks: hence the previous idea, impureEnvVars = [ ("GITHUB_TOKEN_" + foo) ];
<clever> and then nix-daemon would use that value for the builder
<clever> i was under the impression that the nix-build command would read the NIX_CURL_FLAGS from its env, then forward that to nix-daemon over the unix socket
<clever> dmj`: that better not be a real token :P
<clever> so the hash doesnt need to be incremented
<clever> so any change to rev causes the name to not match, and it forces a re-download
<clever> tnks: i think fetchFromGitHub prevents this issue by naming the derivation "${name}-${rev}.tar.gz"
<clever> so it wont notice the hash is wrong
<clever> the problem there, is that if the hash and name match something you already have downloaded, but the url points to a newer file, it wont try to re-download
<clever> dtz: i always increment a random number near the end of the hash
<clever> tnks: you could also do impureEnvVars = [ ("GITHUB_TOKEN_" + foo) ];
<clever> tnks: sure
<clever> which prevents that issue
<clever> LnL: because fetchurl is fixed-output, it ignores the paths of all of its inputs
<clever> not a problem
<clever> which would trigger mass-rebuilds
<clever> otherwise, the source code fetched by fetchurl will change its path every time curl updates
<clever> yeah, it would have been like this for ages
<clever> and if name isnt set, its the filename at the end of the url
<clever> the path that fetchurl returned should have depended purely on the sha256+name that you gave to fetchurl
<clever> and then downloaded it with fetchurl?
<clever> in what case have you noticed that?
<clever> dtzWill: and the other instances of privateNetwork in the same file
<clever> dtzWill: every build goes into its own private network namespace, with only an lo interface
<clever> dtzWill: normal derivations are, but fixed-output derivations are not
<clever> tnks: for attributes within a fixed-output derivation, you can change them all you want without impacting the hash, but those attributes are readable in /nix/store/
<clever> tnks: for NIX_CURL_FLAGS, it bypasses all hashing entirely, but there is a single value for the entire nix-build call, covering all derivations
<clever> havent looked into what default.target covers
<clever> keith_analog: it only runs on bootup if you make it part of the multi-user.target
<clever> tnks: the hash in $out is based purely on the outputHash attribute, which must use the algo in outputHashAlgo
<clever> tnks: the $out hash of a fixed-output derivation ignores its build-time dependencies
<clever> thats easy to convert
<clever> tnks: oh right, thats an older version from before i told dmj to switch it to impureEnvVar
<clever> oh, i had that problem just a day ago, thought it was just me mis-using something
<clever> tnks: this new instance of callPackage will search in self first, then pkgs
<clever> but you need to copy ./foo/default.nix -> foo.nix, and update the src directive to re-pin a new version
<clever> tnks: now it will decide between ./foo/default.nix or ./foo.nix, allowing more complex changes to be made in foo's dir
<clever> foo = if (builtins.pathExists ./foo) then (callPackage ./foo {}) else (callPackage ./foo.nix {});
<clever> another way of doing it:
<clever> and because of how nix works, it will re-check for ./foo each time you eval the nix expressions, and if it doesnt exist, it will always return the same foo with no changes to the hash
<clever> this will override the src if ./foo exists, and just leave foo un-altered if it doesnt
<clever> tnks: foo = if (builtins.pathExists ./foo) then (foo.overrideDerivation (old: { src = ./foo; })) else foo
<clever> tnks: it should be possible to conditionaly do that, one sec
<clever> yeah, just define every dependency in nix, and fetch each private one with code like that gist
<clever> oh, and the curl method from dmj's gist doesnt support submodules
<clever> ah, having nix fetch a pinned version of dependency xyz, i see
<clever> branches and tags to flag versions that are known-good
<clever> ah
<clever> its just a small project, 2 people with push access on a private repo
<clever> so it never has to download private things at build-time
<clever> tnks: in my project, there is only 1 private repo, so i can just "git pull" manualy, and run nix-build in its root dir
<clever> tnks: i am currently writing a testframework in lua, i feel i dont need types that badly :P
<clever> tnks: and now hydra can fetch that input from a private repo, and no build will ever have access to the keys
<clever> tnks: the hydra case is a bit simpler for small numbers of things, you can define an input for hydra as "git@github.com:owner/project.git" and then just run ssh-keygen as the hydra user
<clever> yep
<clever> even if somebody does manage to steal the token, its just read-only
<clever> i believe you can then make a dedicated github account for that, and grant it read-only access to your projects
<clever> tnks: the function in the gist will use an auth token made on github to grant curl access to private repos
<clever> tnks: i prefer to not give the build users access to the private keys themselves, and there is an alternate solution, one sec
<clever> there is an open issue to put agent forwarding directly into nix
<clever> tnks: so i need an socat instance that annonimizes the unix socket connections
<clever> tnks: the reason for that, is that the git clone runs as nixbld1, and the ssh-agent thinks somebody is trying to steal access to the agent
<clever> tnks: in this case, its creating a new unix socket at /tmp/hax, and it will proxy all connections to the unix socket $SSH_AUTH_SOCK
<clever> tnks: its like cat, but it supports tcp sockets, udp sockets, unix sockets, programs, fifo's, and normal files
<clever> tnks: and if you set the url to "git@github.com:user/project.git"; that also applied to github
<clever> tnks: with these commands, you can fetch any private git repo, and fetchgitPrivate will have access to your ssh agent
<clever> tnks: do you want to fetch the private github from a hydra based build, or via nix-build?
<clever> Kendos-Kenlen: and the nixos testcases just run a full test as a qemu vm, and basicaly "compile" the test results when nix tries to "build" the test derivation
<clever> Kendos-Kenlen: any kind of test you can wrap into a nix package can also be handled by hydra
<clever> tnks: will this be working under hydra or nix-build?
<clever> joachifm: of note, mu is broken in master due to webkit stuff, i last checked a few hours ago
<clever> release channels*
<clever> ah yeah, the channels never get darwin builds, only nixpkgs-unstable has darwin
<clever> steveeJ: builtins.readFile is the first thing that comes to my mind
<clever> not sure
<clever> dannyg: that can be tracked in /proc/meminfo but i cant remember which field it is
<clever> dannyg: ive noticed similar when i ran nixos from a uSD card in a usb adapter, abnormaly low write speeds combined with 16gigs of ram, it can buffer pretty massive amounts of data in ram
<clever> dannyg: and when the backlog has gone thru fully, the sync will return, having done exactly what its meant to do
<clever> dannyg: maybe it isnt actualy hanging, but rather, there is a large back-log from rsync, and the sync from nixos-rebuild is waiting for that backlog to clear
<clever> dannyg: not sure then, sounds like an issue within the nfs config
<clever> dannyg: and backup-tud is your only nfs mount?
<clever> nwspk: -p can also accept multiple arguments at once
<clever> strange, what about "strace sync" ?
<clever> dannyg: and it lists the nfs mount?
<clever> dannyg: as a quick check, does "df -h" hang?
<clever> dannyg: because the mount was configured to never fail and to retry forever
<clever> dannyg: 99% of the time, the connection to the nfs server has died, and the kernel is just waiting forever for a reply
<clever> pkg-config --cflags gdk-pixbuf-2.0
<clever> nwspk: it likely also has pkgconfig and the same issue, all glib stuff is designed like that
<clever> nwspk: what does "pkg-config --cflags glib-2.0" output?
<clever> dannyg: i usualy just check the process tree when i see something complex hang, have a look at "ps -eH x"
<clever> nwspk: you need to add -I${glib.dev}/include/glib-2.0 to the include path, -p pkgconfig and "pkg-config --cflags glib-2.0" will find it for you
<clever> nwspk: glib is trying to prevent the kind of collisions that nix is designed to avoid, but it also breaks the logic nix adds to help make things work nicely
<clever> nwspk: glib.h is in include/glib-2.0, but only include is in the include path
<clever> nwspk: ah, i see the problem
<clever> /nix/store/dcha55b8i6xrrbx7bdzv5vakchhf4cvq-glib-2.50.2-dev/include/glib-2.0/glib.h
<clever> nwspk: what did you then run under that nix-shell ?
<clever> dannyg: i think he heard, lol :D
<clever> or just never allow hydra failures, lol
<clever> magnetophon: nix saves all logs to /nix/var/log/nix
<clever> magnetophon: and in the case of things you had built locally and didnt catch the log of, "nix-store -l /nix/store/foo"
<clever> magnetophon: you can look that log up on hydra.nixos.org
<clever> and it still worked, so nobody noticed
<clever> it causes the nix package to contain only static libraries
<clever> one of the json libraries ive used before, renamed the static and dynamic library flags, and silently ignored the old ones
<clever> they can depend on eachother via callPackage as if they where all in nixpkgs
<clever> the main use i have for that, is when i'm making a dozen inter-connected derivations
<clever> steveeJ: yeah
<clever> updated the gist
<clever> oops, the with is wrong
<clever> steveeJ: the callPackage made on line 3 is a new instance of callPackage, that will first search in self, then search in pkgs
<clever> steveeJ: one sec
<clever> steveeJ: thats what newScope is for
<clever> steveeJ: in an attrset you have a reference to?
<clever> gchristensen: i may also be able to answer some arm related questions
<clever> it has spent the last 2 hours on a linux 4.4 kernel
<clever> i have my hydra setup to pre-build a few base packages, and it can take a week to catch up after nixpkgs changes
<clever> yeah, that will probably take a while
<clever> yep :)
<clever> smw_: if you switch over to setting nixos-config=/etc/nixos/configuration.nix in $NIX_PATH, then you can easily override it with -I nixos-config= at any time
<clever> smw_: yeah, that would explain everything
<clever> /nix/var/nix/profiles/per-user/root/channels/nixos:nixos-config=/etc/nixos/configuration.nix:/nix/var/nix/profiles/per-user/root/channels
<clever> [root@router:~]# echo $NIX_PATH
<clever> smw_: on nixos, that path is supposed to be set via $NIX_PATH
<clever> smw_: that is causing it to 100% ignore the sd-image file
<clever> smw_: NIXOS_CONFIG is your problem, that has higher priority then -I nixos-config
<clever> smw_: :p is a bit too verbose, try with just "config.system.build"
<clever> smw_: try running "env > /tmp/a" in tmux, and then "env > /tmp/b" outside, then "diff -u /tmp/a /tmp/b"
<clever> dannyg: ah, yeah, nfs is often a pain like that
<clever> gchristensen: its an internal reference within nixpkgs, would be more complex to find that
<clever> you can move it back after
<clever> smw_: moving the .git out of nixpkgs temporarily may
<clever> smw_: oh, and because its a git clone, that includes the .git dir and all history
<clever> smw_: yeah, it hash to hash tne entire copy of nixpkgs
<clever> smw_: when the above finishes, does it show more?
<clever> smw_: thats interesting, how much ram and swap do you have?
<clever> LnL: yeah
<clever> i believe NIX_LINK has no effect on nix-repl
<clever> smw_: exactly what arguments did you run nix-repl with and what was the value of $NIX_PATH?
<clever> config.system.build
<clever> jsut pastebin everything in build
<clever> what does it show?
<clever> yes
<clever> every standard test said it should have been able to read them, and even running cat as the bind user could read them
<clever> i recently had to deal with apparmor blocking bind from reading zone files
<clever> which appears to be the + version
<clever> globin: i was testing the one from boot.loader.grub.memtest86.enable = true;
<clever> globin: i think i saw one, i had narrowed it down to hardening flags, but forgot to comment on the issue
<clever> globin: on 4 different systems, with radicaly different hardware, and even qemu
<clever> globin: on nixos-unstable, after running for about 2 hours, it finds a problem around 256mb into the ram
<clever> spacekitteh: what if you try restarting the bind service with "systemctl restart bind" ?
<clever> spacekitteh: i cant think of anything else to check, but something is clearly wrong for that defunc to stay around
<clever> spacekitteh: only thing i can think of then is to give memtest86 a run and see what it finds, oh, and the memtest on nixos is broken
<clever> slower, but that also works
<clever> and just checks it all, nix-store --verify --check-contents
<clever> ah, that command expects no arguments
<clever> going entirely by memory
<clever> spacekitteh: and nix-store --verify --check-contents $(nix-store -qR /nix/store/barq9wf0rm61zphrmpd1v29sgd7pch0d-bind-9.10.4-P6)
<clever> spacekitteh: nix-store --verify --check-contents $(nix-store -qR /nix/store/qczk8fm9jj9npmxrkg5r3skfb05p6ahs-unit-script)
<clever> hyper_ch: once i fixed the domain in the config, it worked perfectly in seconds
<clever> the problem appears to be 2 things, named crashed, and bash hasnt called waitpid() to collect the child
<clever> spacekitteh: can you pastebin /nix/store/qczk8fm9jj9npmxrkg5r3skfb05p6ahs-unit-script/bin/bind-start ?
<clever> so the named crashed, but the bash script bind-start hasnt reaped the dead children
<clever> lol
<clever> can you pastebin "ps -eH x"
<clever> it claims to be running normaly, but if all you see if a defunct, then it somehow crashed
<clever> into a pastebin
<clever> then bind isnt even running, journalctl -u bind -f -n 200
<clever> spacekitteh: ps aux | grep named
<clever> spacekitteh: sudo netstat -anp | grep named
<clever> dannyg: that might be it, try rebooting
<clever> spacekitteh: systemctl status bind
<clever> then thats why it isnt working
<clever> tcp6 0 0 :::53 :::* LISTEN 6109/named
<clever> spacekitteh: ok, so it wants to use localhost as the only dns server, netstat -anp | grep 53, is anything listening on port 53?
<clever> dannyg: ah, nothing found, any different if you run that as root?
<clever> spacekitteh: irc is tcp, so it cant be disconnected by changing dns, but it will loose the ability to re-connect by name
<clever> steveeJ: self and newScope are extra stuff, if you wanted to add another level to its search scope
<clever> dannyg: what does this return? "ls -l /proc/*/fd/* | grep gc.lock"
<clever> steveeJ: pkgs.callPackage will search pkgs for unspecified stuff, and "callPackage = pkgs.newScope self" will search self, then pkgs
<clever> spacekitteh: i havent used dnscrypt, but i can help debug it more, what does /etc/resolv.conf contain?
<clever> spacekitteh: oops, wrong s name
<clever> steveeJ: i have 2 dns servers running with bind on nixos, no issues on them, though one is on 16.09, and another is an older copy of unstable
<clever> steveeJ: callPackage will find unspecified arguments in pkgs., and it adds .override
<clever> LnL: ive even seen that "sudo nix-store --delete" will fail 100% of the time, because sudo is "using" the path your trying to get rid of
<clever> LnL: an open shell can block nix-store --delete