2017-02-23

<clever> MichaelRaskin: better open a new https://github.com/NixOS/nixpkgs/issues/4491
<clever> probably has to be aligned to a certain offset at least
<clever> then you dont need special tools to review it
<clever> commit the latex and a nix file for building it into a pdf
<clever> id say, no binary files in git, period
<clever> and git doesnt have any way to diff the binary in a meaningfull way, so nobody can tell what happened
<clever> MichaelRaskin: many binary files like pdf have dead space that you can insert the junk into without any effect
<clever> copumpkin: let me try a few things...
<clever> --export and --import might help here
<clever> copumpkin: so if you can change the state directory when the store gets instantiated, you can make it run against a different db.sqlite
<clever> MichaelRaskin: id think git blame would still trace it back to you
<clever> MichaelRaskin: hmmm, if you can swap out a single file within the tree, it would have to be a hash you previously commited, and nobody has since modified
<clever> copumpkin: so even if somebody swaps out a commit before mine, the git log will claim i just undid your changes, enless you can collide a whole file
<clever> gchristensen: each commit is a snapshot of the entire state of the tree at that point in the commit
<clever> gchristensen: ah, one thing about git, is that it doesnt store differences
<clever> src/libstore/local-store.cc: string dbPath = dbDir + "/db.sqlite";
<clever> copumpkin: yeah, db.sqlite is the biggest issue, i can check the source to see what turns up
<clever> copumpkin: neat, i could see this being of use for osme of my past projects
<clever> copumpkin: aws has fpga services?
<clever> copumpkin: i have read the source for switch-to-configuration before
<clever> LnL: and i can also see it being usefull as a tar replacement, when you dont care about uid/timestamp, but do care about reproducible hashes
<clever> LnL: yeah, you can now unpack any .nar.xz to a non-standard location for debug, without root
<clever> copumpkin: a secondary option is curl http://cache.nixos.org/foo.nar.xz | unxz | nix-store --restore
<clever> copumpkin: its pretty much just tar -c and tar -x
<clever> copumpkin: nope
<clever> copumpkin: did you see what i said about nix-store --dump and --restore lastnight?
<clever> copumpkin: morning
<clever> ah, that also works
<clever> shanemikel: my <nixpkgs> resolves to commit f66d782
<clever> [root@amd-nixos:~]# realpath $(nix-instantiate --find-file nixpkgs)
<clever> /nix/store/18riqb6diaxk5a5bxx5hzh2pyc0jfvv7-nixos-17.03pre99759.f66d782/nixos
<clever> copumpkin: boom, i can now copy the closure of things into a given directory, though it completely ignores db.sqlite
<clever> copumpkin: this will serialize a into a NAR, then deserialize it back into b
<clever> $ nix-store --dump a | nix-store --restore b
<clever> copumpkin: oh, i just had an idea on something you can use

2017-02-22

<clever> i saw it somewhere, months ago
<clever> i dont see an example to copy from /nix/store to /mnt/nix/store, but i assume its in there somewhere
<clever> and your the very first comment on the gist, lol
<clever> ah, examples are in the new ui gist
<clever> copumpkin: isnt clear what format it wants the uri to be in
<clever> copumpkin: yeah
<clever> copumpkin: and now things turn into voodoo, nothing references any function in this file!
<clever> copumpkin: and a random usage example: https://github.com/NixOS/nix/blob/master/src/nix/copy.cc#L46-L49
<clever> copumpkin: ok, this is part of the codepath: https://github.com/NixOS/nix/blob/master/src/libstore/store-api.cc#L692
<clever> copumpkin: yeah
<clever> class LocalFSStore : public virtual Store
<clever> class LocalStore : public LocalFSStore
<clever> BinaryCacheStore is a recent addition, that treats the binary cache as another store, allowing you to just throw copy-closure logic at it, and get substitutes
<clever> both of those require /nix/store to be at /nix/store/
<clever> while applying some permission restrictions
<clever> and then nix-daemon just internaly maps everything to the LocalStore instance inside itself
<clever> RemoteStore is for non-root access, proxying everything via a unix socket to nix-daemon
<clever> LocalFSStore may also be related
<clever> when nix commands are ran as root, all operations go thru an instance of LocalStore, that just directly manipulates db.sqlite and /nix/store/
<clever> let me dig thru the source...
<clever> though maybe builds do work, via normal sandboxing?, or it can build on the host, and copy over
<clever> it cant do builds via that, but it can do the bulk of the operation
<clever> from what i remember, you can just tell nix-store to operate on a normal nix store, that is currently mounted at /mnt/nix/store, and it will just prepend /mnt/ to all paths when doing binary-substituion, and accessing db.sqlite
<clever> ah
<clever> i dont remember the syntax, but nix gained support to do limited operations against a storepath not at /
<clever> there is a new option i was thinking of, that might play into things
<clever> copumpkin: this allows the chroot env to access the entire store of the "host" (which will be the sandbox of the nix build in your case)
<clever> copumpkin: oh, another trick nixos-install uses, NIX_OTHER_STORES and a bind-mount to /mnt/root/nix of the host nix
<clever> yeah, i believe exportReferencesGraph is just runtime
<clever> copumpkin: because of these 2 lines, the stdenv will be in the runtime closure of the test vm sandbox, which means it wont have to download the stdenv when rebuilding things under a vm
<clever> copumpkin: ah, it sounds like you need to include stdenv in the runtime closure?
<clever> *looks*
<clever> you can do the same by putting the wrapper inside make-initrd.nix
<clever> and i dont like the idea of hiding logic inside all-packages.nix
<clever> makeInitrd fakes the same API, by putting a wrapper function into all-packages.nix
<clever> yeah
<clever> copumpkin: in the case of fetchurl, its a lambda that takes stdenv+curl, then it returns a 2nd lambda that wants the url+hash
<clever> copumpkin: thats a hack! lol
<clever> probably the later
<clever> copumpkin: and the way you call make-squashfs.nix is also a bit odd, rather then being 2 lambda's like fetchurl and friends, its a single lamba, and it has to be called in a more specialized way
<clever> copumpkin: but make-squashfs.nix is weird, in that it always strips the /nix/store/ path from things, and expects the FS to be mounted to /nix/store/
<clever> copumpkin: pkgs.makeInitrd is pretty equal to the others, the contents attr lets you set non-store things, like /init being a symlink to the storepath, then it just gets the entire closure automagicaly
<clever> copumpkin: i have used the squashfs and initrd scripts a few times
<clever> copumpkin: ah

2017-02-21

<clever> ndowens08: what about just normal fetchFromGitHub, like the gist i linked?
<clever> i was double-checking against the 16.09 copy by mistake
<clever> ah
<clever> ndowens08: that error message includes output from git, but fetchFromGitHub NEVER uses git, despite its name
<clever> ndowens08: something doesnt match up
<clever> ndowens08: https://gist.github.com/cleverca22/5b581f24f2362ad5a12f313b26f13eed line 15 i believe will do what you wanted
<clever> ndowens08: line 18-21 doesnt look right, and what is the exact error msg?
<clever> ndowens08: and the nixos firewall only blocks incoming, it never touches outgoing traffic
<clever> ndowens08: can you gist your nix expression?
<clever> to get around certain locations being read-only
<clever> threshold: and i believe its using the PGDATA env variable to change where it looks for config files
<clever> threshold: ah, let me see
<clever> no need to touch nix-env or systemPackages
<clever> and it will be installed and configured for you
<clever> threshold: set services.postgresql.enable=true; in configuration.nix and run nixos-rebuild switch
<clever> threshold: what are you trying to get done?
<clever> threshold: just use sudo to get root
<clever> threshold: ~/.nix-profile is entirely managed by nix-env, and you would need to make a package describing the change and nix-env -i it
<clever> gchristensen: i sleep when you least expect it :P
<clever> gchristensen: lol

2017-02-20

<clever> romildo: the command will be available, but not its headers or libraries
<clever> romildo: yeah
<clever> and header files are never installed globally
<clever> romildo: it wont be available inside nix builds, and it wont be able to find libraries
<clever> romildo: you want nix-shell -p llvmPackages.llvm
<clever> romildo: nix is designed such that you never need to install compilers like that
<clever> Judson: and if your building something with nix, you can just pass ${builtins.storeDir} to configure at build-time
<clever> Judson: if you can eval a nix expression, this gives it to you?
<clever> nix-repl> builtins.storeDir
<clever> "/nix/store"
<clever> Judson: there is another way, one min
<clever> yeah, and no hashes on that
<clever> i see, its using perl to parse the reference graph
<clever> oh
<clever> copumpkin: then it has enough to import a db backup (from exportGraph?) to get the db.nix updated
<clever> copumpkin: saw the latest comment on the issue, i believe nixos-install just uses rsync and nix-store -qR to get the closure of nix into the chroot
<clever> copumpkin: only thing i can think of is -qR that destroys the tree, and --query --tree that leaves you with ascii-art

2017-02-19

<clever> but i believe nix-shell and nix-build have been rewritten recently
<clever> dont know
<clever> endformationage: but you can use -I nixpkgs= or NIX_PATH like gchristensen said, to change where <nixpkgs> leads
<clever> endformationage: nope, it ignores -E and the path you give it (it should probably error with invalid combinations)
<clever> and you can shove whatever nix expression you want in foo
<clever> if you -p foo, then it will essentialy do nix-shell -E 'with import <nixpkgs> {}; runCommand "dummy" { buildInputs = [ foo ]; } ""'
<clever> the -p flag is very weird
<clever> endformationage: -p always uses <nixpkgs>

2017-02-17

<clever> shlevy: ah neat
<clever> a fix i came up with ages ago, was to buildEnv the inputs together, then use one -I for that
<clever> and all of the -I's for perl can easily overun it
<clever> the #! line has a max size in the kernel
<clever> i just remembered an issue perl has though
<clever> ah
<clever> Baughn: it doesnt appear to do anything with PERL5LIB at build-time
<clever> top-level/all-packages.nix:10146: buildPerlPackage = callPackage ../development/perl-modules/generic perl;
<clever> Baughn: i havent done much with perl, never even see a Makefile.PL before
<clever> Baughn: that could be improved with a setup hook, that scans $nativeBuildInputs for things in /lib/perl and adds them to $PERL5LIB
<clever> MP2E: thats usualy what hydra saves you from, if your on nixos-unstable
<clever> nix-repl> builtins.typeOf "foo"
<clever> "string"
<clever> ndowens08: what command are you currently using and what are you trying to test?
<clever> Judson: does the shell.nix import <nixpkgs> ?
<clever> ndowens08: the build-vm has a read-only store by default
<clever> Judson: what is in default.nix?
<clever> and then its env vars get set inside the shell
<clever> the env attribute is meant to have nix-shell directly ran on it, via nix-shell -A rubyEnv.env
<clever> buildCommand*
<clever> and with an invalid buildEnv, it cant
<clever> Judson: oh wait no, thats env = rubyEnv.env; which means it has to build that env first
<clever> Judson: that is copying every attribute from the env, including the build command
<clever> Judson: how are you trying to inherit it?
<clever> either keeping the current grub, or blowing it away via nixos-in-place mechanics (running the nix build of grub-install)
<clever> just edit the existing bootloader to make the recovery image the default
<clever> and there is another trick, for cases where kexec isnt in the kernel of the target
<clever> so you just give nixops root access to any linux machine in any datacenter, and it just works
<clever> one thought i had, is to add the kexec stuff to nixops
<clever> yeah
<clever> after that, you just need to ensure nixos always reboots upon any boot failure
<clever> and in either case, kexec the chosen nixos image
<clever> contrapumpkin: but if you do give it the right network packets, you get a boot menu over the network
<clever> contrapumpkin: in theory, i can make a program that runs on bootup, within the initrd, that will wait ~30 seconds for certain network packets, and if it gets none, it boots the default OS
<clever> contrapumpkin: i also then had another idea on how to further improve the usability of nixos in remote locations
<clever> then recover it yourself
<clever> and if you enable ssh in that recovery image, you could ask the datacenter to pick it from grub for you
<clever> contrapumpkin: now you can basicaly boot the install CD at any time
<clever> contrapumpkin: and another idea i had recently, just put this initrd in /boot, and give it a grub.cfg entry
<clever> contrapumpkin: so if you just reboot without formating things, the previous OS boots back up
<clever> contrapumpkin: the kexec works entirely in ram, without any changes to the MBR or bootloader
<clever> yeah
<clever> if i do get in, i can just stop the timer
<clever> contrapumpkin: but a couple extra lines of nix later, i have a systemd timer that reboots back to the original OS every hour
<clever> contrapumpkin: the datacenter i had to deal with lately lacks dhcp, so i had to put static ip config into this file, but if i get it wrong, i cant remotely reboot
<clever> contrapumpkin: line 53-60 of configuration.nix is another addition i did recently to deal with crappy network configs
<clever> contrapumpkin: yep
<clever> contrapumpkin: and within 2 minutes, its running full nixos from ram, you can ssh into that, and wipe the hdd, then do a normal nixos-install
<clever> contrapumpkin: the idea, is that you upload this tarball to any server, unpack it to /, and then just run /kexec_nixos
<clever> contrapumpkin: if you build the kexec_tarball attribute as shown in line 1 of the config, you get a tarball containing a shell script, kernel, and initrd, and the initrd contains a full nixos build
<clever> contrapumpkin: have you seen my kexec tricks with nixos?
<clever> instead of searching right, then left, it evals the entire // operator, and makes a new attret, with references to the values from before
<clever> that might even be faster, but i believe it currently doesnt search, but rather, creates a new attrset with the result of merging
<clever> it overwrites, so a right to left search works
<clever> ah, for the // case, yeah
<clever> value*
<clever> bennofs: i believe it needs to eval both of the // arguments, to find the list of keys, and know which side to get the valid from
<clever> gchristensen: dont think so, you wont even be able to access the .override attribute
<clever> i get a bit OCD about knowing how things work, so ive read relatively large chunks of the nixpkgs and nix source
<clever> now each nixpkgs has its own overrides and config
<clever> yeah, like i just typed
<clever> import <nixos-unstable> { config = { allowUnfree = true; }; }
<clever> just specify its own config as a new attrset
<clever> the overrides for the main nixpkgs are causing that
<clever> oh
<clever> oops, import <nixos-unstable> { config = config.nixpkgs.config; }
<clever> you must import <nixos-unstable> { config = nixpkgs.config; }
<clever> when you imported a 2nd nixpkgs, it didnt get nixpkgs.config
<clever> contrapumpkin: ah right, so the hash will depend on the scripts in here, and which version of tar+gzip+bzip that nix was built against: https://github.com/NixOS/nix/blob/master/corepkgs/unpack-channel.nix
<clever> --export, maybe
<clever> so no ssh and no --import
<clever> a lot of things are restricted under the context of a builder
<clever> ah
<clever> contrapumpkin: nix-copy-closure will just copy storepaths from one box to another, including all dependencies
<clever> contrapumpkin: what about just using nix-copy-closure?
<clever> i would expect that to work, the same
<clever> can you pastebin the default.nix?
<clever> is there a shell.nix?
<clever> and how are you calling nix-shell?
<clever> strange
<clever> ben: and nix-shell will also inherit variables from the parent shell, which leaks stuff like this, --pure stops that
<clever> ben: compare the output of "env | grep PKG" in the shell and build
<clever> or, just copy all of nixpkgs and modify it directly
<clever> then you can customize the module
<clever> or copy the entire module, rename it, and put it in imports
<clever> i think all you can do is serviceConfig = mkForce { ... }; to replace the entire thing
<clever> yeah
<clever> mbrgm: due to how the serviceconfig works, no, the entire serviceConfig is one option
<clever> avn: more about, the xml must be invalid, to be valid
<clever> avn: so you are forced to do description = "para1</para><para>para2";
<clever> avn: one weird thing ive discovered, the entire description is pasted inside a <para>${description}</para>, and it doesnt allow nested para's
<clever> yeah
<clever> yeah, that haproxy config has zero typechecking, so you wont know if its going to break until its too late
<clever> to allow custom stuff they missed
<clever> in some things like nginx, there are a lot of nixos options to map it about, but it also has extraConfig at each level
<clever> so it just needs an option for each
<clever> in the case of my vpn, there is no config file, just a couple commandline arguments
<clever> so it cant break at runtime because you made a typo
<clever> mbrgm: i prefer to map them all to seperate options in nixos, it enforces types and a valid config
<clever> glib and glibc are different
<clever> and glib didnt have a static output, so i had to make an override to rebuild glib
<clever> symphorien: this is how i build a static qemu

2017-02-16

<clever> so youll need to add a few }; and }
<clever> thomassgn: http://pastebin.com/HCwLhErm line 7, i truncated the file
<clever> thomassgn: you need to add it as an override in config.nix for nix tools to know it exists
<clever> you are free to augment that automation as you need
<clever> modify the postInstall that deletes them, to instead move them to $dev/lib/
<clever> if they are tiny, there is no real cost to having them
<clever> i would put those into $dev/lib/
<clever> ah
<clever> ?
<clever> viric: and then if something wants the statics, it can depend on SDL2.static
<clever> viric: if you do that, 99% of the nixpkgs builds will ignore it, so it just sits in the binary cache and costs almost nothing
<clever> viric: i would add a static output, and mv all of them to $static/lib/
<clever> viric: oh, that will do it, heh
<clever> [clever@amd-nixos:~]$ nix-store -r /nix/store/h03qmjqfczaadhrlhpbr9prm0fh4pv6b-SDL2-2.0.5
<clever> yeah
<clever> philipp[m]: and next time systemd boots that container, it uses that version
<clever> philipp[m]: the declarative containers dont have their own profiles, so when you build the host, it also updates the active nixos for every container in the systemd config
<clever> mojjo: not really, you would need some ugly hacks with unionfs, or just use something raid capable (mdadm, zfs, lvm)
<clever> viric: nix-repl also helps a lot with this: http://pastebin.com/fc0EDyaX line 33&34
<clever> on nixos-unstable, it does
<clever> on 16.09 it doesnt
<clever> ah
<clever> nix-repl> SDL2.out
<clever> «derivation /nix/store/x21l6wp6qvbz262avzjdziz4n18cigpi-SDL2-2.0.4.drv»
<clever> viric: i only see a .out on SDL2, no .outputs, so it has 1 output, thats it
<clever> viric: checking .outputs should tell you what that package has
<clever> viric: and sometimes a static
<clever> viric: ive seen bin, dev, lib, and dbg within nixpkgs
<clever> viric: and derivations are free to make their own ones up, i have one for logs and static binaries
<clever> viric: every string listed in the outputs attribute becomes an output
<clever> simpson: either M$ thinks their cloud is bigger then the sum of every single user, or the NSA wants help spying on things, lol
<clever> and then only the official one will remain, and only if you update
<clever> so any pre-existing skype clients are going to break
<clever> and ive heard rumors that skype is shutting down the p2p network and going purely "cloud" based
<clever> yeah, tox can only talk to other tox users
<clever> mojjo: toxcore has been working to take over the skype users, https://tox.chat/
<clever> declarative containers just update every single container, since the host depends on the container builds
<clever> or one container may wind up on a month old nixos-unstable, because you forgot to run updates on it
<clever> simpson: so you need to "nixos-rebuild --upgrade switch" every container seperately
<clever> simpson: yeah, they all share the store, but the "active" nixos build is going to be entirely out of sync
<clever> with declarative containers, nixos-rebuild on the host updates the build of every container at once for you
<clever> philipp[m]: imperative containers gain the use of "nixos-rebuild switch" within them for faster updates, but they also have some cons: each container has its own channel, and you now have to nixos-rebuild every container one by one
<clever> mojjo: yeah, the unfree error should tell you what contents it needs
<clever> philipp[m]: the only way to update the declarative containers is to restart their systemd units on the host, which does a full container shutdown, and then bootup
<clever> mojjo: nix-env and nix-build only ever use ~/.nixpkgs/config.nix
<clever> mojjo: configuration.nix is only used for nixos-rebuild, nix-env never reads it
<clever> simpson: and also, it forces a linux-32bit build, even on a mac system!!
<clever> simpson: callPackage_i686 forces skype to use 32bit libraries and to build in a 32bit way
<clever> hyper_ch: some of them are also r/w and can be changed at runtime
<clever> hyper_ch: in here is every option to every module loaded into the kernel, including modules that linked in at compile time
<clever> /sys/module/sg/parameters
<clever> /sys/module/sg/parameters/allow_dio
<clever> hyper_ch: find /sys | grep module | grep para | head
<clever> about the only way to break nix and make it disobey the nix expressions is to forcibly edit existing files in /nix/store/
<clever> its probably using the real configuration.nix, and ignoring your changes entirely
<clever> ndowens08: and the option is nixos-config, not nix-config
<clever> ndowens08: nix always uses a new directory in /tmp for every build, no mater what the sandbox is set to
<clever> eacameron: nixpkgs will handle ./configure and everything automaticaly
<clever> eacameron: if the source is available, start with just a derivation containing a name and src = fetchurl and see what happens