2017-06-21

<clever> Infinisil: there is also import from derivation, just make a derivation that contains a nix file, then import it
<clever> Infinisil: you can use builtins.readFile to read a file and then manipulate it
<clever> simpson: and you are also reporting the set of object hashes you currently have in your ipfs store
<clever> simpson: so anybody that knows your pubkey can find your ip, and track you, world-wide
<clever> simpson: one issue ive noticed with ipfs, is that you have a long-term keypair, and while online, you are storing a pubkey=ip record in the DHT
<clever> they would be using the cached build most likely, and not notice
<clever> disasm: any time something low-level like glibc or gcc gets changed, nixpkgs will mass-rebuild as a side-effect, and then things like lxc might get noticed
<clever> yeah, about 28,000 jobs on nixpkgs
<clever> BlessJah: more that the tests got more strict
<clever> Sonarpulse: i have been testing out the PR and github status stuff in hydra, and it could easily cover a lot of this
<clever> nix changed the rules
<clever> so the CI at merge time would have never caught it
<clever> but a recent security update in nix broke lxc, after it was merged
<clever> i believe in this case, the old nix was allowing the setuid
<clever> BlessJah: yeah
<clever> BlessJah: only a subset of the builds (see above link) are required for the channel to update
<clever> BlessJah: there are currently 700 failing builds in nixpkgs, the channel would never update if it waited for 100%
<clever> BlessJah: most things in nixpkgs arent flagged to block the channel updates
<clever> it has since been updated to 2.0.8
<clever> BlessJah: hydra did catch it: http://hydra.nixos.org/build/54439612/nixlog/1
<clever> BlessJah: no, setuid binaries arent allowed in the store, and the store should be fully immutable
<clever> yeah, it may need something like kvm to even be able to talk to virtio channels
<clever> so you can just -I nixos-config=foo.nix, override the kernel, and then run it outside the test framework for more control
<clever> digitalmentat: that module is also used by "nixos-rebuild build-vm"
<clever> it uses the 9p driver with the virtio backend
<clever> a random note from 2013, no clue what its about, lol
<clever> 14 20:58:22<@clever> 4g bare, 16g bolt
<clever> and has a total of 9000 lines!
<clever> said irc channel has logs going back 4 years on this machine alone
<clever> i have an old irc channel i never took off auto-join, that i just paste random junk to
<clever> heh
<clever> pie_: i need to keep a better todo list, lol
<clever> via Sphinx
<clever> yeah, llvm_4 on linux depends on SQLAlchemy
<clever> ah, and the llvm attribute is 3.9.1
<clever> copumpkin: strange, the dep-tree on my hydra says it does affect llvm
<clever> | | | | | | +---/nix/store/rxzcwnbfb4i94qijz40236gfrpscas2j-python2.7-Sphinx-1.5.2.drv
<clever> | | | | | | +---/nix/store/rl22ayalrc93s8jp71rzqkadf6w5gbg3-llvm-4.0.0.src.tar.xz.drv
<clever> maybe that was a fluke related to a nearby change
<clever> FRidh: ive noticed it affecting llvm and a few other things
<clever> ah
<clever> Sonarpulse: yeah, it may also work on other platforms that lack signalfd
<clever> so now try to ls it
<clever> the makepanda directory does exist
<clever> justanotheruser: try inserting an "ls -ltrh" call before the python3, and then use it to look around
<clever> Sonarpulse: and i think the sigint from ctrl+c may go to several threads, so a different thread could catch it, then write a byte to an internal pipe
<clever> Sonarpulse: at least on linux, there is also a seperate stack for signal handlers: https://www.gnu.org/software/libc/manual/html_node/Signal-Stack.html
<clever> and the daemon will enforce that the <hash> in /nix/store/<hash>-name.drv matches the hash of its own contents
<clever> the eval is always done as your user, and it pushes the contents of .drv files up to the daemon
<clever> the api has changed
<clever> and yeah, the old nix-repl doesnt appear to be compatible
<clever> ah, nvm then
<clever> Sonarpulse: nix-shell -p '(nix-repl.override { nix = nixUnstable; })' i think
<clever> Sonarpulse: but you may need to build the repl against unstable
<clever> goibhniu: its what cd's into the directory made by unpackPhase
<clever> goibhniu: that cd happens after the unpackPhase has finished
<clever> goibhniu: its near the cd into $sourceRoot
<clever> goibhniu: just run "nix-store -r /nix/store/f3wan424j5p0a06p02byvn4gq5g2x234-stdenv" and you can fetch that exact build from the binary cache
<clever> raunov: instead of mounting the installer over virtual usb, it just copies the whole thing to ram from another linux
<clever> raunov: something else you might be able to use is this: https://github.com/cleverca22/nix-tests/tree/master/kexec
<clever> that was an "a or b" question, not a yes/no question
<clever> raunov: ah, is it emulating a cdrom or a usb stick?
<clever> raunov: how did you flash the usb with the iso?
<clever> you may just need src = ./.;
<clever> what does a whl file normally contain?
<clever> given what you have said, it should work with just src = ./panda3d-1.10.0-cp35-cp35m-linux_x86_64.whl;
<clever> what error does it fail with if you dont quote it?
<clever> the path also needs to not be quoted
<clever> and then srcSHA wont be needed either
<clever> and if you use ./ it will be relative to the file that path is inside
<clever> in nix, you can have bare paths, which must start with either ./ or /
<clever> thats how most nix stuff handles this situation
<clever> you might be able to do src = ./panda3d-1.10.0-cp35-cp35m-linux_x86_64.whl;
<clever> so it cant access /home
<clever> yeah, the build runs inside a chroot like sandbox
<clever> srcURL = "file:///home/justanotheruser/panda3d/panda3d-1.10.0-cp35-cp35m-linux_x86_64.whl";
<clever> ah
<clever> justanotheruser: can you gist the .nix file nix-shell was loading?
<clever> justanotheruser: what command did you run to trigger that error?
<clever> tommyangelo[m]: yeah, if you build once with -I nixpkgs=/etc/nixos/nixpkgs, then it will use the value in the config, which is the same
<clever> tommyangelo[m]: you would need to use nix.nixPath in the configuration.nix to control what lands in $NIX_PATH after its built
<clever> tommyangelo[m]: nope
<clever> symphorien: tommyangelo[m]: i believe it needs to be -I nixpkgs=/path/to/repo
<clever> nixy: yep
<clever> nixy: personally, i just delete that file and add its entries to configuration.nix on most systems, i know what i'm doing and dont need it to auto-generate everything for me
<clever> nixy: you also have the option to run nixos-generate-config again after installing, to update that config
<clever> wb?
<clever> i need to start a blog
<clever> kk
<clever> i just took the kernel+initrd from the ipxe netboot, and ran it with kexec instead
<clever> ah, then you may have seen this before: https://github.com/cleverca22/nix-tests/tree/master/kexec
<clever> eacameron: have you seen how kexec works?
<clever> then change the boot order back
<clever> remotely reinstall
<clever> and using a modified netboot image, you can boot it into a nixos installer with ssh pre-configured
<clever> then you can just change an entry in the db, to mess with the boot order, remotely
<clever> so you could set the systems to always network boot, then boot.php could tell it to boot the local hdd anyways
<clever> ipxe is also capable of chainloading the internal hdd's of a machine, i believe
<clever> this file puts iscsi rootfs support into the initrd
<clever> thats enough to get the kernel+initrd loaded
<clever> so any "dumb" os (grub, dos, and so on) will just work over iscsi
<clever> ipxe has a mode, where it hijacks the legacy bios api for the hdd
<clever> i didnt even enable network support in grub
<clever> but its 100% over the network
<clever> as in, the zvol has a full grub in the MBR, and boots as if it was a hdd
<clever> this line will boot a normal nixos install from a zvol, on my laptop
<clever> and also, if you configure iscsi, you could specialy boot some systems from a different image entirely: https://gist.github.com/cleverca22/75e3db4dabeff05b743e9be77a2341b9#file-boot-php-L6
<clever> and if your internet is fast, you could load some of those files over the web, just beware of mitm
<clever> depending on how many clients boot at once, you may want to put in a load balancer, either inteligent http redirects, or just some dns round-robin
<clever> at this point, the only thing to consider is the network bandwidth when downloading 278mb initd's over http
<clever> you can even download pre-built netboot images from a hydra
<clever> line 82 of the previous file also handles embeding a squashfs with the nixstore into the initrd
<clever> which just loads the kernel+initrd from the current dir
<clever> and that script is handled here
<clever> which itself, could be inside a profile managed by nix-env
<clever> in the case of line 20, it points the client to the file made by https://github.com/NixOS/nixpkgs/blob/master/nixos/release.nix#L106-L127
<clever> (i just hard-coded it all)
<clever> this will decide what os the pc should boot, based on the mac address and your favorite db
<clever> step 4, boot.php (you can just rewrite it into any server-side scripting, or use a static file)
<clever> this contains variable substitutions that ipxe will perform, then it fetches the given url over http
<clever> step 3, the dhcp server now gives a different answer, on line 10
<clever> that will get ipxe running on the client, which will re-query dhcp, and this time, set the user-class to ipxe
<clever> that file comes from the ipxe package (its already in nixpkgs)
<clever> and then lines 8 and 12 together, say to download and execute undionly.kpxe over tftp, from a server at .2.61
<clever> step 2, line 12 of this dhcpd.conf will detect people network booting
<clever> there it is!
<clever> no, not that one either, what was it called, lol
<clever> i have too many gists, lol
<clever> oh, wrong gist, lol
<clever> step 1, enable network booting in the clients, they will ask the dhcp server for the config
<clever> i also have a gist with more info
<clever> eacameron: main cost with this method, is that your entire rootfs is held in ram, and has to download at bootup
<clever> then using a modified form of the above nix-env, you can build that, and keep generations so you can undo
<clever> eacameron: so you could copy this expression, insert a custom module on line 109, and then enable a full gui
<clever> and the entire rootfs is contained within the initrd
<clever> eacameron: the netboot target in here, makes a directory with 3 files, netboot.ipxe, initrd, and bzImage
<clever> ah, i didnt paste one for the other netboot
<clever> oh, wrong cmd
<clever> 2017-06-20 08:19:57< clever> [root@router:/tftproot/try2]# nix-env -p /nix/var/nix/profiles/per-user/root/rpi3-netboot -f not-os/release.nix -A rpi_image -I nixpkgs=./nixpkgs/ --set
<clever> that could just be nfs, and mount the same /home to everything, and let the username do the rest
<clever> would you want the clients to have a full r/w disk, or read-only?
<clever> ah, i had a dedicated zvol for each client, so they could persist changes
<clever> eacameron: i was network booting my laptop and 2 raspberry pi's over iscsi, and i had nixos as the iscsi target at one point

2017-06-20

<clever> and ignores the swap entirely
<clever> it just goes to 0 io/sec, and 100% cpu on every core
<clever> nh2: ive also found that my system prefers to go cpu bound, rather then use the swap i gave it
<clever> in the past, i have tried to turn overcommit off, it broke everything
<clever> yeah
<clever> Mic92: on 64bit systems, haskell will just allocate 1tb of virtual memory and then never talk to the kernel again, lol
<clever> Mic92: oh, and that reminds me, haskell does away with that problem, and the whole mmap vs brk deal
<clever> it would be far faster to just keep that heap in userland, and not bother the kernel every time you free a few kb up for 2 ms
<clever> growing the heap ~4 times, then shrinking it to the original size, and repeating with the exact same sizes
<clever> when the old version version (before that PR) was chewing thru my cpu, strace said it was calling brk() non-stop
<clever> i also noticed when discovering https://github.com/NixOS/nixpkgs/pull/26554 about 2 days ago, that bash (or glibc?) is overly aggressive at shrinking the heap
<clever> so it allows resizing the mapping that is used as heap space
<clever> gchristensen: the brk syscall is used to set the end of the heap, and by tracking where it was ending, you can move it up/down as you grow/shrink the heap
<clever> gchristensen: why cant you operate purely with anon mmap's?
<clever> gchristensen: i have also wondered if brk is even needed anymore
<clever> though i could see char foo[strlen(input)]; being a potential exploit
<clever> that sounds like it would need more then 4kb on the stack as local vars, or an exploit that can apply an arb decrement to the stack pointer
<clever> so if you go byte by byte down the stack, you can never hit the heap
<clever> gchristensen: i would expect the kernel to include some guard pages, something that acts as a limit and will always page-fault/segv
<clever> and another reason to go all 64bit
<clever> gchristensen: oh god, thats such a simple question, lol
<clever> but i did notice that the binary cache didnt have it, and it had to build locally
<clever> i had tested it on cb90e6a036 and it worked
<clever> /nix/var/nix/profiles/per-user/root happens to already be a place that can hold roots, so it saves a step
<clever> avn: which is managed under /nix/var/nix/gcroots
<clever> avn: maybe, it needs to be somewhere that is a valid gcroot
<clever> so it will only ever contain 1 result
<clever> and --set will replace the entire contents of the new generation, with the result of the build
<clever> avn: -f, -A, and -I do the usual things
<clever> avn: the -p flag points it to a custom profile, where it will store all generations
<clever> avn: ah, i was using --set, not -i
<clever> [root@router:/tftproot/try2]# nix-env -p /nix/var/nix/profiles/per-user/root/rpi3-netboot -f not-os/release.nix -A rpi_image -I nixpkgs=./nixpkgs/ --set
<clever> avn: let me check for that example i made a few months back
<clever> avn: i believe if you use nix-env -i -p /path/to/profile, it will automaticaly manage a manifest for you
<clever> nix-shell -E 'with import /home/clever/apps/nixpkgs {}; runCommandCC "shell" { buildInputs = [ stuff ]; } ""'
<clever> or paste the same string together in your own file or -E flag
<clever> gchristensen: so whatever you do to -I or NIX_PATH, must have an effect on <nixpkgs>
<clever> gchristensen: it will just paste together a few strings, including import <nixpkgs>, and then pass it thru -E
<clever> gchristensen: let me find a source reference
<clever> gchristensen: oh, you need to set nixpkgs= in the search path
<clever> avn: the channelname.foo stuff is only ever in nix-env, and it uses .nix-defexpr/channels_root/manifest.nix i think
<clever> avn: ah, i believe that will just use the foo attribute within nixpkgs
<clever> avn: if you do <foo>, then nix will search the foo= entries in NIX_PATH, and also search the foo subdir of anything that lacks a prefix
<clever> srhb: glusterfs builds for me on nixos-17.03
<clever> srhb: which channel are you on?
<clever> srhb: the spew of errors at the end tells you the dependency chain
<clever> i suspect its waiting for every sync to be flushed to all 3 drives
<clever> mine is oddly slow, for some reason my nas with a zfs raid over 3 drives has high latency
<clever> lol
<clever> ah, i was about to make my own
<clever> ikwildrpepper: updated
<clever> ikwildrpepper: what projects do you see on https://hydra.angeldsis.com/ ?
<clever> ikwildrpepper: oddly, my router hydra (it runs a version from several years ago), doesnt have this issue
<clever> yep, pulling that up now
<clever> ikwildrpepper: i think its getting worse, i can now see hidden projects on the homepage of the hydra, without signing in
<clever> gchristensen: in cases like that, i just basic-auth the entire hydra at nginx
<clever> gchristensen: ive found that the hidden jobsets and hidden projects are poorly hidden, you can still see them on half the pages
<clever> hyper_ch: then you dont have that pdf viewer installed
<clever> evince is /run/current-system/sw/bin/evince
<clever> [clever@amd-nixos:~]$ type evince
<clever> turion: you can also just ask bash where it is
<clever> turion: and if you installed it with nix-env, then it will be in ~/.nix-profile/bin/

2017-06-18

<clever> '';
<clever> PS1=things
<clever> so you can just do shellHook = ''
<clever> i think the attribute was called shellHook
<clever> if you put that into default.nix, it will have the same affect as the -p flags
<clever> with import <nixpkgs> {}; stdenv.mkDerivation { name = "foo"; buildInputs = [ abc def ]; }
<clever> ah
<clever> matumental: -p doesnt read default.nix
<clever> and as root, you may need to nix-channel --update
<clever> and you need to not delete those symlinks
<clever> jake_: you need to remove the nixos channel from the non-root acct
<clever> jake_: its complaining that you have a nixos channel on both your user and root
<clever> jake_: you may need to recreate them
<clever> lrwxrwxrwx 1 clever users 44 Oct 11 2015 channels_root -> /nix/var/nix/profiles/per-user/root/channels
<clever> lrwxrwxrwx 1 clever users 46 Mar 7 2016 channels -> /nix/var/nix/profiles/per-user/clever/channels
<clever> $ ls -l ~/.nix-defexpr/
<clever> total 1
<clever> jake_: what is in this dir?
<clever> $ ls -l ~/.nix-defexpr/channels_root/
<clever> and for non-root?
<clever> jake_: what does nix-channel --list say?
<clever> matumental: and make sure you fix the uuid for /boot in /etc/nixos/, so it knows which one to mount
<clever> matumental: you can also "nixos-install --chroot" and then "nixos-rebuild boot" to use the nixpkgs inside the install
<clever> matumental: nixos-install by itself, will use the installers copy of nixpkgs to fix it, which may upgrade or downgrade things temporarily
<clever> it randomly gives ipv6 only replies
<clever> i think its a crappy router
<clever> wait, why is it using ipv6, lol
<clever> download-from-binary-cache.pl: could not download ‘https://cache.nixos.org/1l9chqsl9k3gqvlagbzd3wcvrphvq9j9.narinfo’ (Curl error 7)
<clever> catern: yeah
<clever> dont know, havent done that much with docker
<clever> the result of running which uname at build-time
<clever> ij: or you need to bake the path of coreutils into your build, by either adding ${coreutils}/bin to $PATH, or adding $(which uname) in where you call uname
<clever> ij: you can either rely on the fact that coreutils is always installed, and just expect it to be in $PATH
<clever> ij: at runtime or build time?
<clever> at least under root, the channel name should be nixos
<clever> ah
<clever> what channel came back on its own?
<clever> is this on nixos or another distro?
<clever> tilpner: its to handle creating a default the first time nixos boots
<clever> /home/clever/apps/nixpkgs/nixos/modules/programs/shell.nix: echo "${config.system.defaultChannel} nixos" > $HOME/.nix-channels
<clever> tilpner: nixos will add a default one if the dir is missing on root

2017-06-17

<clever> catern: i heard 80gig per eval
<clever> then when its done, it does a normal rm -rf on trash
<clever> sphalerite: it moves things from /nix/store/ to /nix/store/trash, because directory moves are atomic
<clever> sphalerite: and in the past, i have tried to get gpu passthru under xen to work, but discovered that the host must fully reboot any time the windows reboots
<clever> sphalerite: i have a server in the cloud running win7 to handle some legacy junk that relies on windows
<clever> ben: that is a property that will affect the hash
<clever> lol
<clever> ah
<clever> which is usualy what you need
<clever> i think "$@" handles that?
<clever> yeah, just change the copy it made on boot, and nixos-rebuild
<clever> jake_: and all the stores closed 3 minutes before i started helping you today, lol
<clever> jake_: nixos-rebuild should work inside the container
<clever> jake_: when i get back from a grocery run, i can merge the PR
<clever> typo!
<clever> + cp ${./configuration.nix} /etc/nixos/container-helper.nix
<clever> container-helper.nix has to be copied to /etc/nixos/
<clever> i also just noticed a minor bug you can fix
<clever> edit the file, git add, git commit --amend, git push --force
<clever> you can do it more easily
<clever> either way works
<clever> on the real containers, its just an ordinary file, and the logs go there to die
<clever> jake_: i believe that if kmsg is created as a character device, the guest can write to the real dmesg buffer
<clever> yeah, 116 should set them up, as long as /dev/fd exists
<clever> since its a copy of the store, is much safer to bend the rules and just edit stage-2 after you untar
<clever> inserting calls to env would help track it down
<clever> i cant find them being set anywhere
<clever> good question
<clever> /home/clever/apps/nixpkgs/nixos/modules/system/boot/stage-1-init.sh:logOutFd=8 && logErrFd=9
<clever> jake_: strange, the source i can find says its 8 and 9, not 62
<clever> jake_: oh, i have a thought, let me double-check
<clever> i almost always build with -Q -j8, and then re-build with -j1 to see the error
<clever> jake_: -Q
<clever> sure
<clever> jake_: you need this symlink inside the chroot
<clever> lrwxrwxrwx 1 root root 13 Jan 26 00:18 /dev/fd -> /proc/self/fd