2017-07-17

<clever> dtzWill: the code is designed around a binary cache, it will treat the narinfo url's differently from the nar urls
<clever> and cache.nixos.org is often "slow"
<clever> bennofs: nix will cache things in a sqlite database, but the ttl is fairly short, and it isnt shared between many machines
<clever> bennofs: this program will proxy all requests to the binary cache, and cache the replies in ram
<clever> this is a binary cache cache
<clever> dtzWill: ive also been working on this in my spare time: https://github.com/cleverca22/cachecache/
<clever> qknight: can you link to your service on github?
<clever> qknight: but you need to still pass a nixos-config to it
<clever> qknight: you can also just eval/instantiate -A config.services.nginx.foo to target any option
<clever> qknight: and if you know what attribute the config is at, you can just build/instantiate that directly
<clever> qknight: this would generate a perl script that runs nginx with all of the right args, based on the service config in the configuration.nix in the current dir
<clever> qknight: nix-build '<nixpkgs/nixos>' -A config.systemd.services.nginx.runner -I nixos-config=./configuration.nix
<clever> qknight: no real way, you need to instantiate the top-level of a nixos eval that imports the module
<clever> looks ok to me
<clever> gchristensen: i think each build takes 8 hours

2017-07-16

<clever> yeah, thought you meant the uppercase one
<clever> Wizek__: yeah
<clever> Wizek__: yes
<clever> boomshroom: this guide
<clever> boomshroom: let me check my irc logs
<clever> boomshroom: ah, then you want the linode guide
<clever> boomshroom: as a vm, or as just a dir in the store?, or a disk image?
<clever> i was using my router as an example, and its in dire need of an update
<clever> oh, that sounds better
<clever> Infinisil: nix.extraOptions in configuration.nix
<clever> S0rin: yeah, like LnL said, --add only takes effect after --update
<clever> Infinisil: auto-optimise-store = true
<clever> Infinisil: there is also an option to make nix internally run it after every operation
<clever> S0rin: did you also nix-channel --update as root?
<clever> S0rin: are you checking roots channel list or your own user?
<clever> bennofs: i see an eval from 3 days ago that fully passed, so it should have updated
<clever> gchristensen: bit of a mouthful now, but it looks like the older method will still work, if you dont care about the hash
<clever> Guest73314: create a packageOverride that does nginx = pkgs.nginx.override { withStream = true; };
<clever> but still no evals finishing
<clever> gchristensen: the postgres was dead, but that came up just recently
<clever> gchristensen: ah, so it wasnt really fixed
<clever> gchristensen: strange, no eval has passed in the last 21 hours
<clever> ah, yeah
<clever> ?
<clever> thanks
<clever> niksnut: can i also get an eval started on http://hydra.nixos.org/jobset/nixpkgs/haskell-updates ?
<clever> S0rin: looks like its up now
<clever> about this time yesterday, evals where failing due to low space
<clever> yeah, it probably ran out of disk space
<clever> cherrybl0ss0m_: yeah, thats a possibility
<clever> cherrybl0ss0m_: what does this command say: nix-store -q --roots /nix/store/00fhhmww8gmf6cg55b9f37fzp67wjm4l-diffutils-3.6
<clever> cherrybl0ss0m_: ahh
<clever> cherrybl0ss0m_: what user owns /nix/store/ on that system?
<clever> cherrybl0ss0m_: so the problem is actualy when nix is trying to enforce what your expecting it to enforce
<clever> the error is happening inside the part of nix that ensures everything has an mtime of 1
<clever> ah
<clever> so if root is a member, the system isnt going to survive
<clever> cherrybl0ss0m_: nix will pick a user from that group, then kill -9 every process, and start the build as that user
<clever> cherrybl0ss0m_: that is a very wrong value for that setting
<clever> cherrybl0ss0m_: why is build-users-group set to root?
<clever> cherrybl0ss0m_: can you gist more of the error?
<clever> it shouldnt be trying to change things i think
<clever> can you give more context for the error?
<clever> ah, what is the error?
<clever> which file exactly has that time?
<clever> cherrybl0ss0m_: i think its safe to just leave the mtime like that
<clever> but those shouldnt be that big...
<clever> and the journal
<clever> there still is the postgres db
<clever> LnL: ah, it was out of disk space yesterday
<clever> Mic92: i would generally prefer to run patchSheBangs over the dir the script is in, so it just works all the time

2017-07-15

<clever> so we just need a modified form of that, that can patchelf an arm ghc
<clever> there are already bootstrap ghc's that patchelf the official dist tarballs
<clever> jophish: dont remember if he got it working or not
<clever> Winchell1M: then you can "nixos-rebuild boot" to update the system
<clever> Winchell1M: after mounting everything under /mnt like you did for the install, run the above command to get a shell inside that /mnt
<clever> yeah, common problem, you either need to boot from the installer and use "nixos-install --chroot" or plug in an ethernet cord
<clever> and if wireless.enable = false;, it wont even install wpa_supplicant, so you have no way to manualy start it
<clever> johnw: so you can do it like this
<clever> also, because your doing the substitution at the nix level, src2 doesnt have to be in the attrset
<clever> you need to tell cp to undo that rename
<clever> yep
<clever> what does echo ${src2} say in the configurePhase ?
<clever> try to echo the ${src2} and youll see it
<clever> and if you dont tell cp what the destination name is, it keeps the hash
<clever> ./foo winds up at /nix/store/<hash>-foo/
<clever> the directory gets renamed
<clever> johnw: and if you set it to src, the default unpackPhase will copy it for you
<clever> johnw: any variable you add to a derivation becomes an env variable
<clever> src = lib.cleanSource ./path/to/local/source;
<clever> y
<clever> yeah, and just skip the git layer entirel
<clever> pkgs.fetchgitLocal
<clever> there is fetchgitlocal for that
<clever> a member of the nixbld group
<clever> it doesnt use virtual machines on nixos
<clever> schoppenhauer: are you on a mac?
<clever> schoppenhauer: i cant see any reason why the agent wouldnt work, just start an agent and run ssh-add on a key that has access
<clever> praduca: they are in a file called hackage-packages.nix, its massive
<clever> schoppenhauer: the ssh agent socket handles that, so the build users never need read access
<clever> schoppenhauer: any attempt to do that will result in the key landing in /nix/store and being world-readable
<clever> NVME L2arc, with sata SSD for the main storage
<clever> aristid: too many machines that lack nvme support in the uefi firmware
<clever> and if you GC, it can cease to boot entirely
<clever> if /boot isnt mounted at rebuild time, the system will just rollback all changes every time you reboot
<clever> simukis: this method even works for modules that have been compiled into the kernel
<clever> 48 "spl.spl_taskq_thread_bind=1"
<clever> 47 "zfs.zio_taskq_batch_pct=50"
<clever> 42 kernelParams = [
<clever> simukis: kernel commandline
<clever> simukis: nice
<clever> until somebody upstream changes something
<clever> :(
<clever> jophish: steam has a similiar problem, it needs to use an older glibc, but then the mesa drivers on the host break with that older glibc
<clever> S0rin:, oops
<clever> so: try building with "--option build-cores 1"
<clever> so i havent seen its initrd stuff
<clever> i avoided systemd like the plague before coming to nixos
<clever> systemd and its deps is 85mb
<clever> that might make it more bloated
<clever> systemd isnt in the initrd
<clever> and you may also want to add usb back in
<clever> keep in mind, that mkForce will disable anything zfs was doing to help, so you need to manualy put it back in
<clever> maybe
<clever> boot.initrd.kernelModules = lib.mkForce [ "xhci_pci" "ahci" "zfs" ];
<clever> but you can always use mkForce
<clever> the default is a bit extreme
<clever> simukis: is there anything listed in /etc/nixos/hardware-configuration.nix?
<clever> simukis: most of that usb stuff is so your keyboard continues to work in the initrd, and dm_mod is because lvm isnt optional
<clever> simukis: which ones?
<clever> gchristensen: and sourcing /etc/profile from ~/.bashrc does make it work
<clever> gchristensen: aha, even sourcing /etc/bashrc from ~/.bashrc doesnt work
<clever> andreabedini: the old script would use sudo to chown /nix for you, and then do the rest without root
<clever> that explains why tab complete has been wonky
<clever> uhh, what? lol
<clever> lrwxr-xr-x@ 1 root wheel 11B Sep 21 2016 /etc -> private/etc
<clever> yeah, no real way to fix this system wide, other then spamage in every home
<clever> gchristensen: opensnoop confirms, bash only ever tries to use ~/.bashrc, so it requires imperative setup to work
<clever> i think you have to nix-build it, and then it yeah that
<clever> i dont think nix-shell works on that
<clever> gchristensen: aha, opensnoop looks good
<clever> "/nix/store/il00b6zlxfb3201nq07cd54w4p489zjc-nixos-17.09pre110213.01c3847b9c/nixos/pkgs/tools/misc/peruse/default.nix:17"
<clever> nix-repl> peruse.meta.position
<clever> ToxicFrog: nixos-unstable
<clever> gchristensen: at a glance, this looks like it can do almost anything to any process, and its scriptable
<clever> error: attribute ‘unwrapped’ missing, at (string):1:1
<clever> nix-repl> pkgs.peruse.unwrapped
<clever> i dont see an unwrapped attribute on my end
<clever> gchristensen: looks a lot more powerful then strace
<clever> you can also symlink that from ~/.nixpkgs/ so your user just uses it by default
<clever> import <nixpkgs> { config = import /etc/nixos/config.nix; }
<clever> now the config is broken out into a smaller file, that nixpkgs can load on its own
<clever> ToxicFrog: one handy trick, put all of your overrides into /etc/nixos/config.nix, then in configuration do nixpkgs.config = import ./config.nix;
<clever> only nixos loads configuration.nix
<clever> correct, nixpkgs will never load configuration.nix
<clever> ToxicFrog: if you pass it {}, then it will load ~/.nixpkgs/config.nix automatically
<clever> i'll google it some more
<clever> gchristensen: do you happen to remember how to trace syscalls and such on darwin?
<clever> gchristensen: and on the mac, it ignores /etc/bash.bashrc
<clever> olejorgenb: ah, wasnt expecting that
<clever> gchristensen: i also see a /etc/bash.bashrc in the source, hmmm
<clever> ToxicFrog: pkgs.peruse.<tab><tab>
<clever> gchristensen: yeah, this kind of thing is crazy
<clever> gchristensen: why is bash doing this??? :O
<clever> 1090 ->>>>>>> ~/.bashrc if we are a top-level shell. */
<clever> 1089 /* If we were run by sshd or we think we were run by rshd, execute
<clever> so, it only ever runs ~/.bashrc, and /etc/bashrc is just an addition by the distro, (via /etc/profile) ?
<clever> gchristensen: after a glance at the bash manpage, i see no mention of /etc/bashrc!
<clever> handy
<clever> ah
<clever> but like this, the bash survives
<clever> \-+= 36207 admin bash -c echo foo ; /Users/admin/.nix-profile/bin/pstree
<clever> [clever@amd-nixos:~/iohk/nixpkgs]$ ssh admin@de302.macincloud.com 'echo foo ; /Users/admin/.nix-profile/bin/pstree'
<clever> as in, either sshd directly ran pstree, or the shell just ran execve without a fork
<clever> if i run a bare command with ssh, like "ssh darwin pstree", i see no bash between sshd and pstree
<clever> ok, now thats fun...
<clever> lol
<clever> now it needs to load coreutils before it even knows what version of nixpkgs to load
<clever> and my bash scripts used readlink -f to find their own location, to correctly work with relative paths
<clever> until you nix-shell -p coreutils
<clever> i also recently discovered, darwin lacks realpath, and "readlink -f" doesnt work
<clever> the root problem is that bash only loads ~/.bashrc when ran over ssh, hmmm
<clever> it already loads /etc/profile and /etc/bashrc by proxy, so it no longer needs ~/.bashrc for interactive shells
<clever> but that script no longer exists
<clever> previously, that mac .bashrc was sourcing a nix script in ~/.nix-profile/
<clever> though..., it might not set NIX_REMOTE=daemon
<clever> and then just that one binary will work without any scripts having been loaded
<clever> gchristensen: if you only want to get build slaves working, you could leak a nix-store symlink into /usr/bin/
<clever> gchristensen: updated the gist
<clever> if i then run bash inside that, it only does ~/.bashrc
<clever> aha, but thats only for the shell launched by sshd
<clever> and ~/.bashrc is ignored
<clever> gchristensen: for an interactive shell, it loads /etc/profile (which sources /etc/bashrc on this machine), then it loads ~/.profile
<clever> odd, /etc/profile is readonly on this box, an contains references to nix
<clever> but keep in mind, those echo's can break scp, so undo them when your done
<clever> i find it helps to spray every profile and bashrc with an echo
<clever> the shell skips some files when doing non-interactive
<clever> grahamc: ah, so its not just my setup
<clever> gchristensen: does your new darwin thing put nix into PATH early enough for "ssh darwin nix-store --version" to work?
<clever> gchristensen: does your new darwin thing put nix into PATH early enough for "ssh darwin nix-store --version" to work?
<clever> srhb: so the result of runCommand will wind up in the store (with world-readable keys), and you then need to use nix to insert that storepath somewhere
<clever> srhb: if sandboxing is on (it should always be on), there is no way for runCommand to access /var/
<clever> and if you add something to systemPackages that has a $out/etc/xdg/autostart, the same thing will work
<clever> ixxie: one-shot systemd unit?
<clever> and then everything else is up to u-boot
<clever> and typically, you put a combined SPL + u-boot at that offset of the block device
<clever> basicaly, it will either boot over usb, or it will execute a block at a hard-coded offset on the sd/mmc/nand/spi flash
<clever> ah, found the specs for the allwinner: http://linux-sunxi.org/BROM
<clever> sphalerite[m]: i have thought of that for remote nixos machines
<clever> TimePath: is this an allwinner cpu your on?
<clever> i believe android does it by having a thin bootloader before the real one, that can reflash over usb, and must never change (i think its a rom in the cpu die)
<clever> ah, but you would need a 3rd partition for syslinux and that config
<clever> but how will the system know which boot to use?
<clever> LnL: filterSources refuses to accept a storepath as an input
<clever> LnL: the problem me and taktoa[m] where trying to solve, is to take an unpacked .tar.gz, and then copy every single .c and .h out of it, into seperate storepaths, content addressed
<clever> LnL: yeah
<clever> do you have some example code or input and what you want done?
<clever> sphalerite[m]: the problem i was having then, is that i needed to force a file in the store to be re-copied to the store, as a content-addressed entry (fixed output)
<clever> sphalerite[m]: i dont think there is any way to read a binary string, i was thinking of a new primop to fix the problem there
<clever> yeah, and thats not really user friendly
<clever> and because of bash, you cant just nix-build -A "foo.bar" or -A foo."bar.baz"
<clever> sphalerite[m]: i ran into a problem a few days ago, builtins.readFile cant read a binary
<clever> sphalerite[m]: there is also a lib.optionalAttr that handles the if for you
<clever> { "foo.bar" = "baz"; }
<clever> key names can also contain a .
<clever> LnL: i think i saw that in the source somewhere, it allows you to omit things conditionaly
<clever> olejorgenb: id expect PKG_CONFIG_PATH to be blank by default

2017-07-14

<clever> fetchurl: yeah, factorio uses a pretty weird and unique fetching system
<clever> nh2: using nix-store --delete, you should be able to remove the failed build, and then try it again
<clever> nh2: and the build "passed" even with a fatal error
<clever> nh2: ah, so it sounds like one of the machines had already "built" a copy of that file, and it didnt have to copy it over from the good machine
<clever> nh2: what about the output of "nix-store -l /nix/store/<foo>"
<clever> nh2: what happens if you run nix-store --query --hash on that path, on both machines?
<clever> fetchurl: its in the changelogs for nix
<clever> works on nixos-unstable
<clever> [clever@amd-nixos:~]$ nix-prefetch-url '<nixpkgs>' -A hello.src
<clever> downloading ‘http://ftpmirror.gnu.org/hello/hello-2.10.tar.gz’... [0/0 KiB, 0.0 KiB/s]
<clever> not sure, try it and see what happens
<clever> with the new nix-prefetch-url -A, it will rename the output after the download is finished, so it always lands in a path matching its hash, even if the hash isnt what nix was expecting
<clever> and nix will enforce the data matching that hash
<clever> with normal fixed-output downloads in nix, you tell nix what the hash of $out will be ahead of time, and nix computes $out based on that hash, and ignores all build inputs
<clever> oops
<clever> so when you run the nix, it wont do a 2nd download
<clever> joepie91: that will download it, and then rename its entry in the store, to match the real hash
<clever> fetchurl: there is also a new thing, nix-prefetch-url '<nixpkgs>' -A hello.src i think it was
<clever> ah
<clever> then let nix-build fail
<clever> so its invalid, and wont match up with the old file
<clever> i usualy just increment a random number in the hash
<clever> LnL: that one always messed with my tab-complete, and my mention filter only triggers if my name is at the very start of a line
<clever> ison111: and you must also add systemd.service.httpd.wantedBy = lib.mkForce [];
<clever> ison111: yeah, that enable must be set to true
<clever> you must set enable = true; and add the line i gave above
<clever> so there is nothing to start
<clever> ison111: if you set enabled = false;, then nixos wont even create the service
<clever> if you remove enable, then the systemd service wont even be created
<clever> and add the line i gave above
<clever> keep the existing config, including the enable line
<clever> ison111: to undo that, you would need to do systemd.service.httpd.wantedBy = lib.mkForce [];
<clever> ison111: and the code behind services.http.enable sets wantedBy for you
<clever> your setting services.http, not systemd.services.httpd
<clever> can you gist your configuration.nix?
<clever> ison111: systemctl show httpd.service | grep WantedBy
<clever> so if another module already set wantedBy, that will also take effect
<clever> and keep in mind, nixos will merge all the modules together
<clever> then it shouldnt start on bootup
<clever> just dont even set wantedBy
<clever> ison111: this line makes the service run on bootup, just leave it out: https://github.com/NixOS/nixpkgs/blob/release-17.03/nixos/modules/services/networking/ssh/sshd.nix#L247
<clever> and it wont run on startup
<clever> ison111: yes, just dont set wantedBy
<clever> there is also "nixops deploy --build-only", which will build everything, but not actualy deploy it to a remote system
<clever> can you gist every file involved?
<clever> where did you write that?
<clever> and if its not adding the options, configuration.nix will fail
<clever> if you rename the main thing under options.services, then you can use it in the configuration.nix file
<clever> thats the simplest way to confirm its loading the file
<clever> put a syntax error in and confirm if "nixops deploy --evaluate-only" fails
<clever> ah
<clever> what is your custom module going to do?
<clever> inspect the output from --build-only, and confirm if it created the right files
<clever> there is a nixops deploy --build-only and --evaluate-only
<clever> yeah
<clever> schoppenhauer: nixops also takes a list of modules when you create a deployment
<clever> schoppenhauer: configuration.nix is another module
<clever> schoppenhauer: you can add an imports list to any nixos module, to extend the module list
<clever> Filystyn: that will drop you into a root shell, bypassing all login prompts
<clever> Filystyn: if your locked out, you can try adding "init=/bin/sh" to the kernel commandline in grub
<clever> Filystyn: run passwd as root
<clever> ah
<clever> dont know, havent actually used that backend