2016-11-29

<clever> nix-build '<nixpkgs/nixos>' -I nixos-config=./configuration.nix -A config.system.build.toplevel
<clever> and you can pre-build the images and embed them into the host as well
<clever> if the host has something, it just copies it over, if not, it tries the binary cache, and just compiles whatever is missing
<clever> Zvieratko: then it just runs nix-build under the chroot
<clever> Zvieratko: it will basicaly rsync a copy of nix (and its dependencies) into the root, and register them as valid in nix.db, then configure it so it can leech from /nix/store of the host (like a 2nd binary cache)
<clever> Zvieratko: the nixos-install script does something similar
<clever> Zvieratko: https://github.com/cleverca22/nixos-installer is the github page for it
<clever> and then it just goes whatever the files told it to do
<clever> Zvieratko: one mode of operation would be to give it a tar or git url, containing the full config and partition layout
<clever> Zvieratko: that is one of the things i had planned
<clever> Zvieratko: so you can just boot an image that has this project running on bootup, and a central box can take over and make it install things
<clever> Zvieratko: one of the side-projects i'm working on is an html5 gui for nixos-install, with the option for an RPC
<clever> manveru: and done!
<clever> { a = "b"; c = "d"; }
<clever> nix-repl> :p builtins.listToAttrs (map (e: { name = builtins.elemAt e 0; value = builtins.elemAt e 1; }) [["a" "b"] ["c" "d"]])
<clever> manveru: map, almost done this example
<clever> Zvieratko: ah, neat, sounds similar to something i have planned
<clever> and other stuff has to actualy bring the machine up
<clever> Zvieratko: physical machine support is thru the targetenv of "none", which just ssh's into a given machine and applies the config
<clever> manveru: yeah, one sec
<clever> then nixops will deploy the final config
<clever> same config for all
<clever> so you could use the example.nix image to format and install a bare-bones nixos
<clever> Zvieratko: http://nixos.org/nixops/
<clever> yeah, nixops is better at pushing the config out to a machine, once it has nixos
<clever> treat the image from example.nix more as an ISO then the final product
<clever> yeah, the image would do that
<clever> so a system boots the image over the network, and the image will automaticaly format the host, run nixos-install, and reboot
<clever> Zvieratko: with some minor tweaks, you could use that image as an installation media
<clever> Zvieratko: this would generate an image based on the that config
<clever> Zvieratko: https://github.com/cleverca22/nix-tests/blob/master/example.nix and nix-build '<nixpkgs/nixos>' -I nixos-config=./example.nix -A config.system.build.example
<clever> Zvieratko: you can run nixos-install from a script, and there are utils in nixpkgs to build an image that has a script built into it
<clever> Zvieratko: the main way i can think of right now, is to setup something that will install a base image (with ssh and some credentials to login), and then use nixops with the targetenv of "none" to deploy
<clever> ixxie: only gentoo does that, heh
<clever> ah, i didnt notice that typo in the original paste
<clever> qknight: the left version will call fetchurl on an attrset, the right version returns an attrset with fetchurl set to another attrset
<clever> ixxie: its what creates the main pkgs attribute set
<clever> ixxie: that contains nearly every package in nixpkgs
<clever> 2016-10-23 13:03:28-!- Lethalman [~lethal@185.51.73.86] has quit [Ping timeout: 244 seconds]
<clever> ixxie: i saw him here a few days ago
<clever> qknight: you should be able to just run nix-instantiate directly on that file
<clever> i often say, "luke, use the source", because the docs can lie :P
<clever> i have noticed some man pages like nix-collect-garbage refer to arguments like --print-live, which only work on nix-store --gc
<clever> mbrgm: yeah, the manual is just manualy edited
<clever> (the nixpkgs manual)
<clever> mbrgm: mostly in the source file lib.nix, and nixpkgs
<clever> ixxie: it was partialy deleted when gnome's default version got updated
<clever> ixxie: yeah, id call this a bug
<clever> ixxie: so my first guess is pkgs.callPackage <nixpkgs/pkgs/desktops/gnome-3/extensions/workspace-grid.nix> {};
<clever> ixxie: a quick grep says that it isnt referenced by any other file in nixpkgs
<clever> srhb: so you can do { foo = let bar = true in VALUE; } or { args }: let bar = true in VALUE
<clever> srhb: i think it can go before any value
<clever> echo-area: usualy you just ignore the contexts, and nix does its magic for you
<clever> echo-area: not sure
<clever> qknight: and the txt of the same name shows you the expected output
<clever> qknight: https://github.com/cleverca22/nix-tests/blob/master/bare-env.nix this is a sample i made that shows sandboxing in action
<clever> echo-area: and then if something uses findfile to find it, nix would automaticaly download the given revision of nixpkgs
<clever> echo-area: http://pastebin.com/w4WZyy0Z is something i was experimenting with yesterday, and i could put pkgs2 into the nix search path as-is
<clever> echo-area: but in some weird cases, the path list given to findFile may contain a derivation
<clever> echo-area: <nixpkgs>/default.nix doesnt actualy contain a derivation, it returns an attribute set full of derivations
<clever> but normaly, those contexts get routed back to another derivation, and become its build-time dependencies
<clever> and in this case, findFile will try to compile everything the string depends on, at line 811
<clever> what you cant see, is that it also references /nix/store/ay9q989f0qwvrjjnn6pq5skc69rlmkmc-hello-2.10.drv
<clever> so if i make a string like "${pkgs.hello}", that has context that depends on a specific build of hello, but when cast to a string, it looks like "/nix/store/cjdj0d229a4d8cr872di5yvyrm65qg83-hello-2.10"
<clever> basicaly, every string in nix has context attached to it, which references which derivations the string may depend on
<clever> PathSet context; is another weird thing in nix that can be non-obvious
<clever> in this case, findFile requires that every entry is an attrset
<clever> but yeah, it is valid for a list to have a mix of types, [ 5 "10" { a=true; } ] is a list of 3 items
<clever> its forcing it from a lazy thunk to an attribute set, which can throw an exception if it isnt an attrset
<clever> state.forceAttrs(v2, pos);
<clever> echo-area: state is an internal part of the evaluator, its not one of the variables passed to findFile
<clever> 5 + "10" is invalid, so we dont have another javascript on our hands
<clever> numbers wont just magicaly turn into strings, but argument types dont have a defined type either
<clever> echo-area: nix-repl> builtins.findFile builtins.nixPath "nixpkgs"
<clever> echo-area: it wants a list and the thing to find

2016-11-28

<clever> nh2_: ah
<clever> nh2_: and does this work without root? :S
<clever> if (pivot_root(".", "real-root") == -1)
<clever> nh2_: my gut says chrome uses that, but i havent confirmed if it works without root
<clever> nh2_: 2257 will also try to create a new lo interface within the new network namespace
<clever> nh2_: dont remember
<clever> nh2_: and more logic at 2166
<clever> nh2_: 1853 is where it starts the sandbox mechanics
<clever> ah
<clever> nh2_: sounds like you need a "make clean" rule to delete the state
<clever> gchristensen: yeah, like the snmp case
<clever> so, it compiles itself to open "unknown" at runtime
<clever> nh2_: the nix sandboxes lack all variants
<clever> nh2_: it tries to detect the name of /etc/mtab at compile time, (it varies from distro to distro)
<clever> nh2_: though i have run across an impurity in snmpd, that was tricky to track down
<clever> nh2_: i generaly feel that sandboxes can be used to force impure stuff to break, as an alarm, but once you have make the expression pure, it will build the same with or without
<clever> nh2_: i have heard that chrome uses the same path to lock its threads down, but i didnt think it could do that much without root
<clever> nh2_: even without root?
<clever> nh2_: you can see the expected output in the txt file by the same name
<clever> nh2_: this is a nix file i wrote that shows the process namespaces working: https://github.com/cleverca22/nix-tests/blob/master/bare-env.nix
<clever> JonReed: substituteInPlace $pkg "/usr/bin/env python" ${python2}/bin/python
<clever> JonReed: only other thing i can think of then is substituteInPlace
<clever> JonReed: if its missing, then runCommand is the one with gcc
<clever> JonReed: it should be in pkgs.runCommandCC, but maybe your channel doesnt have it yet
<clever> it needs root to do a lot of the sandbox stuff
<clever> dont think there is any way
<clever> then the daemon will do all the root tasks for your non-root users
<clever> nh2_: run nix-daemon as root, and export NIX_REMOTE=daemon
<clever> JonReed: try it with runCommandCC maybe?
<clever> nh2_: its an option on nix-store, the man page is wrong
<clever> JonReed: and you ran it on $out/bin ?
<clever> JonReed: can you pastebin the resulting output after it built?
<clever> JonReed: ah, it does specialy handle env correctly
<clever> ah, it is in buildInputs
<clever> 2016-11-28 11:53:01 < clever> JonReed: which will search $PATH at build time (put python2 into buildInputs)
<clever> it will patch everything in that dir
<clever> JonReed: one minor detail, patchShebangs needs a directory, not a file
<clever> JonReed: make a nix package for it that will download and patch it
<clever> JonReed: which will search $PATH at build time (put python2 into buildInputs)
<clever> JonReed: if it already has #!/usr/bin/python, you can use patchShebangs
<clever> JonReed: use #!${pkgs.python2}/bin/python
<clever> kbwt: and they will always get the version the ask for
<clever> kbwt: and unlike other distros, you can depend on a&b, without a&b being "installed" globaly, so nothing else will be able to use a&b, enless they also depend on it directly
<clever> kbwt: all dependencies are resolved by evaluating the nix expressions, which is much faster on nix then on gentoo
<clever> and different versions with different build config also get unique prefixes
<clever> kbwt: nix solves most issues, by not really installing things in the traditional way, every version of every package gets a unique --prefix
<clever> taktoa: this one is just the ansii color code escape, but it probably allows all of unicode
<clever> taktoa: non-printing characters are in this line: https://github.com/cleverca22/not-os/blob/master/stage-1.nix#L11
<clever> earldouglas: and the default builder isnt involved, so it couldnt even if it was an attr
<clever> earldouglas: its still called src, but it is not a src attribute inside mkDerivation, so stdenv wont do anything with it
<clever> earldouglas: http://pastebin.com/fAj9R2ku like this
<clever> earldouglas: and you can put that fetchurl into a let block, then refer to it in line 7
<clever> i would just not even use src, or mkDerivation
<clever> ah
<clever> earldouglas: what are you trying to do?
<clever> earldouglas: but stdenv will unpack whatever you put into $src
<clever> earldouglas: fetchurl doesnt unpack by default
<clever> and manage the whole list in git
<clever> and it lets you atomicly modify the list of packages
<clever> it uses attribute paths, so it wont switch to the dev version on its own
<clever> nix-env -iA nixos.mystuff will install(or upgrade) all of my stuff
<clever> vdemeester: http://pastebin.com/waUNDPYw in ~/.nixpkgs/config.nix
<clever> vdemeester: i prefer using a buildEnv, for package management
<clever> i tend to just delete the file and manage that manualy in configuration.nix, so i know exactly what is changing
<clever> normaly, you can re-run nixos-generate-config on a running machine, and it will update hardware-configuration.nix to match whatever you currently have mounted
<clever> its more of a user config problem then a bug in nixos
<clever> one person garbage-collected that generation, and then it lost the ability to boot :P
<clever> and from then on, it always rolls back to the last config that made it into /boot, on every restart
<clever> this happens every now and then with nixos users, /boot gets removed from configuration.nix by mistake
<clever> you have been updating the entries file on the rootfs, not the one the bootloader used
<clever> yeah, so you need to mount sda2 to /boot
<clever> can you mount sda2 to /mnt and run "find /mnt" and pastebin that output?
<clever> what is sda1, sda2, and sda3?
<clever> do you have a dedicated boot partition?
<clever> i dont see anything mounted to /boot
<clever> joncfoo: can you pastebin the output of "mount" and "lsblk" ?
<clever> ah, not grub then, but same conditions still apply
<clever> you picked exactly 44?, or just "latest"?
<clever> is /boot mounted correctly?
<clever> either you selected 42 from grub when booting, or something has glitched out
<clever> you are currently running from generation 42, and there are 2 new ones, 43&44
<clever> joncfoo: can you pastebin the output of this?
<clever> ls -l /run/current-system /nix/var/nix/profiles/system*
<clever> joncfoo: one sec
<clever> joncfoo: is virtio in that file?
<clever> joncfoo: and its configured via /etc/modules-load.d/nixos.conf which nixos should generate
<clever> joncfoo: i think a systemd unit called systemd-modules-load handles loading boot.kernelModules
<clever> Dezgeg: i think i found my boot problems, ARM_APPENDED_DTB is enabled when it shouldnt be
<clever> shanemikel: when stdenv is in use, the builder is set to https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/default-builder.sh
<clever> it should be in the nixpkgs manual
<clever> shanemikel: on line 6-8, you can see the effect of the process namespace, the builder believes that bash is pid 1, and the ps aux is pid 2
<clever> shanemikel: and the output when running it: https://github.com/cleverca22/nix-tests/blob/master/bare-env.txt
<clever> shanemikel: here is an example of a bare-bones derivation, that makes no direct use of stdenv: https://github.com/cleverca22/nix-tests/blob/master/bare-env.nix
<clever> shanemikel: buildInputs are loaded by setup.sh, which ran by nix
<clever> Biappi: if its using autoconf, you can do buildInputs = [ autoreconfHook ]; to generate one
<clever> so the tar.bz2 you pointed it at lacks a configure script
<clever> checking one of your pastebins i see this: no configure script, doing nothing
<clever> ahh
<clever> Biappi: you probably want an override instead
<clever> Biappi: i see where your problem is now, your missing all of jacks buildInputs, so you basicaly have to rewrite the jack package
<clever> make a new nix file, with only lines 4-11 of the current one, and run nix-shell on that
<clever> oh, and your override wont be there
<clever> that will give you a shell for building jack, rather then a shell for building calf
<clever> Biappi: ah, then you will probably just want nix-shell '<nixpkgs>' -A libjack
<clever> Biappi: what is the contents of default.nix?
<clever> Biappi: so nix-shell is failing at the dependency building, and re-running it wont help
<clever> Biappi: the problem is not in the derivation nix-shell was ran against
<clever> Biappi: after the failure, you can re-run nix-shell and it will restore the env
<clever> nix-shell will load everything in buildInputs for you
<clever> Biappi: if its already in buildInputs, then you dont need -p libjack
<clever> Biappi: nix-shell -E 'with import <nixpkgs>{}; (callPackage ./default.nix {}).overrideDerivation (old: { nativeBuildInputs = old.nativeBuildInputs ++ [ libjack ]; })'
<clever> Biappi: or use an override
<clever> Biappi: modify the nix file to add that to buildInputs
<clever> Biappi: -p and passing files in dont work together

2016-11-27

<clever> nix-shell -p patchelf
<clever> lf
<clever> and you usualy dont want to install things like patche
<clever> the kernel injects linux-vdso.so into every process, so the userland can get the best performance without having to care about what the cpu is
<clever> and you need to use the right one for syscalls, to get the best performance
<clever> basicaly, every cpu has a different trick to switch to kernel mode
<clever> its a weird hack to make syscalls perform better
<clever> nope, linux-vdso.so is part of the kernel itself
<clever> yeah
<clever> not what i was thinking of then
<clever> ToxicFrog: nixos or another distro?
<clever> based on buildInputs
<clever> LD_LIBRARY_PATH is only setup like that inside nix-shell
<clever> so it breaks for everybody, rather then just those who forgot to install X
<clever> and also why nix is designed to avoid looking in global paths
<clever> yeah, i'm not sure on that part either
<clever> yep
<clever> iMatejC: just following along with what your saying
<clever> so you will want a second variable made via mkLibraryPath for those, and then use wrapProgram
<clever> ToxicFrog: i think dlopen can only be solved via LD_LIBRARY_PATH
<clever> iMatejC: ah
<clever> you must use libPath in --set-rpath
<clever> all that does is set a variable called libPath
<clever> not sure how its finding the others, but xorg.libX11 is the package for libX11
<clever> you must point it to the path of everything
<clever> it will never find "installed" libraries
<clever> for closed-source stuff, buildInputs is only usefull to get things in $PATH at build time
<clever> when compiling stuff from source, the rpath is just set correctly by the compiler (via buildInputs)
<clever> ToxicFrog: you need to --set-rpath with a list of the runtime deps
<clever> but if you get it into nixpkgs, i can add it to my jobsets and pre-build it
<clever> ahh, then you will need a more customized kernel
<clever> yep, that build is following the nixos-unstable-small channel
<clever> i should have the rpi fork of the kernel in there, but when i tested it recently i couldnt get it to boot
<clever> iMatejC: its following nixos-unstable-small if you want to match your nix-channel up
<clever> iMatejC: with this, you can leech things ive built on my hydra
<clever> iMatejC: let me fetch my binary cache info
<clever> iMatejC: i havent finished patching install.in yet, but that should let you do the curl install | sh thing on an armv7
<clever> iMatejC: i also have some arm stuff on my hydra
<clever> simpson: but it might be simpler to just use a proper config+option pair like everything else, if your only going to have a single cool.nix loaded
<clever> simpson: and then inside that, { foo }: { pkgs, config, ... }: { normal junk }
<clever> simpson: i suspect you can do imports = [ (import ./cool.nix { foo = "bar": }) ];
<clever> glines: gcc.cc.lib
<clever> glines: best case, it has a CLI mode, worst case, it needs a full x server to unpack the rest
<clever> glines: i see xorg functions in the new binary
<clever> XftFontOpenPattern
<clever> eek
<clever> so you needed to patchelf the binary in ram
<clever> without using the execve syscall
<clever> what i believe UPX does, is it unpacks the 246mb copy to ram, then internaly re-executes that version
<clever> yeah, now you can patchelf that version
<clever> glines: running upx -d on it turns it into a 246mb elf dynamic elf file
<clever> glines: yep, its a valid UPX self-extracting-archive
<clever> testing TeensyduinoInstall.linux64 [OK]
<clever> [nix-shell:/nas]$ upx -t TeensyduinoInstall.linux64
<clever> [root@amd-nixos:~]# nix-shell -p upx
<clever> glines: i see a upx package in nix!
<clever> glines: if you can install upx, you can use that to extract the files from it
<clever> glines: oh, upx is a special packer
<clever> 64043547 0x3D13A1B Copyright string: " (C) 1996-2010 the UPX Team. All Rights Reserved. $l Rights Reserved. $"
<clever> binwalk sometimes gives false readings
<clever> 54574169 0x340BC59 GIF image data, version "89a", 16458 x 16547
<clever> glines: can you paste a link of where you downloaded that ELF?
<clever> you can then use those with dd to extract the tar from inside the elf, and do sane things with it
<clever> benley: yeah
<clever> glines: it basicaly brute-force searches every byte offset of the file with "file" to find things like a tar inside the ELF
<clever> glines: you might be able to get something out of it using binwalk
<clever> glines: i'm guessing its either some form of weird DRM, or something similar
<clever> glines: i have a feeling they cheated, and made a "static" executable, that opens the dynamic linker anywahs
<clever> glines: that sounds very strange, is there an execve anywhere in the strace?
<clever> glines: what does "file" say when ran on the program?
<clever> and at build-time, it creates those symlinks
<clever> you could then package the launcher, and have it depend on doomrl
<clever> ah
<clever> and if you use $out/bin, then it will be in /run/current-system/sw/bin, which is part of $PATH
<clever> if you put the package into environment.systemPackages, it will wind up in /run/current-system/sw/opt/doomrl
<clever> i would make a proper package for it, and add it to the configuration.nix
<clever> ah
<clever> and by extension, it wont break the ELF file nix isnt aware of
<clever> so nix will never garbage collect zlib
<clever> and as an added benefit, nix believes that the shell script depends on zlib
<clever> without making a proper package
<clever> so i could then use the above gist to re-patch the ELF every time steam updates it
<clever> one case where i can see the above being usefull on its own, one of the games ive played in steam, downloads .exe and ELF files, when running under wine
<clever> this allows you to quickly test things out, either just to make it work, or as a step leading up to making a package
<clever> and if you run that shell script on an ELF file, it will fix the dynamic linker, and make zlib available at runtime
<clever> ToxicFrog: if you run nix-build on that nix file, it will generate a shell script
<clever> ToxicFrog: this is a util i wrote that helps speed up some steps
<clever> ToxicFrog: i also wrote a util for this a few months back, one sec
<clever> ToxicFrog: thats done by the build process for stuff in nixpkgs, but it wont happen to ELF files you just download
<clever> griff_: it might be the cached state of the cache
<clever> griff_: weird, mine just uses the cache automaticaly
<clever> ToxicFrog: programs on nix will never find an so that is "installed", you must give it an absolute path to every library
<clever> JonReed: store it in a packageOverride or nixpkgs.config ?
<clever> ToxicFrog: nix-shell has so source a bunch of bash scripts, a lot of its features are bash functions
<clever> griff_: does it have internet access?