2019-06-07

<clever> o1lo01ol1o: builtins.fetchGit wont get the submodules
<clever> o1lo01ol1o: it just runs a chunk of bash to make something
<clever> > runCommand "name" {} "mkdir $out; touch $out/foo"
<clever> o1lo01ol1o: your only real option is to fetch every repo seperate with builtins.fetchGit, and then piece them together with runCommand
<clever> builtins.fetchGit kills parallelism
<clever> ashkitten: builtins.fetchGit is impure and happens at eval time, pkgs.fetchgit is pure and happens at build time
<clever> o1lo01ol1o: pkgs.fetchgit supports submodules, but not private repos
<clever> o1lo01ol1o: only builtins.fetchGit can use that private key, and that doesnt support submodules
<clever> o1lo01ol1o: yeah, submodules only work nicely on public git repos
<clever> o1lo01ol1o: your in for a world of pain :P
<clever> o1lo01ol1o: public or private?
<clever> tobiasBora: that sounds more like a question that #systemd could answer now
<clever> tobiasBora: not really
<clever> tobiasBora: run `man yes` and try `yes` in a terminal
<clever> Henson: once you get the closure of the new version of nixos in /nix/store, yeah, just update the profile and run switch-to-configuration
<clever> tobiasBora: can you pastebin the contents of that file?
<clever> and other distros expect you to do it yourself
<clever> more that nix reloads it for you, after modifying the files
<clever> tobiasBora: nixos and home-manager automate that step for you
<clever> tobiasBora: i think you need to daemon-reload any time the unit files are modified
<clever> tobiasBora: link each file, or use the anything.nix example above, which only has 1 file
<clever> tobiasBora: and you only need the .service file, and nothing else
<clever> thats the semi-auto part i mentioned
<clever> tobiasBora: you need to reload the user daemon
<clever> colemickens: `route -n` tells it to not
<clever> colemickens: it performs dns lookups on every IP in the routing table
<clever> o1lo01ol1o: it is now working
<clever> o1lo01ol1o: the remote end forced it to be nix-store --serve, so its now waiting for some nix protocol commands
<clever> tobiasBora: shove that whole thing into anything.nix, and then simply `nix-build anything.nix`
<clever> let eval = import <nixpkgs/nixos> { configuration = ./myconfiguration.nix; }; pkgs = import <nixpkgs>{}; in pkgs.buildEnv { name = "things"; paths = [ eval.config.systemd.user.units."pulseaudio.service".unix ]; }
<clever> tobiasBora: the next step would be to switch to a nix expr
<clever> tobiasBora: then you just need to create the symlink in ~/.config/systemd/user/ once, and it will update semi-automatically
<clever> tobiasBora: nix-env -f '<nixpkgs/nixos>' -I nixos-config=myconfiguration.nix -iA 'config.systemd.user.units."pulseaudio.service".unix'
<clever> tobiasBora: you could install it with nix-env, then you get a .service in ~/.nix-profile/
<clever> tobiasBora: so its trying to build a fully working nixos, and the assertions found mistakes
<clever> tobiasBora: you left out the -A i gave above
<clever> o1lo01ol1o: try removing the shell= on the remote build user, also it shouldnt have any extra groups
<clever> o1lo01ol1o: just use .source, dont bother with readFile, https://gist.github.com/o1lo01ol1o/df97356d495a014dc68e9547f78d3f86#file-infra-nix-L30
<clever> o1lo01ol1o: i usually call the build user builder, not hydra-queue-runner
<clever> o1lo01ol1o: what is in the $HOME of this remote user?
<clever> o1lo01ol1o: something feels wrong for nixos to have this issue
<clever> o1lo01ol1o: did you try adding isNormalUser = true; ?
<clever> tobiasBora: yeah, and it will also ensure the Y exists under X first
<clever> o1lo01ol1o: i also suspect your build user has been created with the wrong flags, does isNormalUser = true; fix it?
<clever> o1lo01ol1o: the nice part was optional
<clever> tobiasBora: because the default.nix in nixos looks for <nixos-config>
<clever> tobiasBora: or write a nix expr that imports it and gets several unit files
<clever> tobiasBora: and if you want a single unit file, maybe add -A 'config.systemd.user.units."pulseaudio.service".unit'
<clever> tobiasBora: nix-build '<nixpkgs/nixos>' -I nixos-config=configuration.nix
<clever> but nix-diff cant diff that for you
<clever> colemickens: you can just download it from cachix using `nix-store -r /nix/store/foo`
<clever> tobiasBora: the modules is basically a framework to create a single derivation, that has all of your config embeded into it
<clever> colemickens: nix-diff only works on drv files, not outputs
<clever> tobiasBora: it can help to sprinkle echo's in all of the files, and see which ones trigger when
<clever> so you can force it to always run /run/current-system/sw/bin/nix-store
<clever> o1lo01ol1o: this forces the connection to run a given command, if using a given key
<clever> o1lo01ol1o: you can also force it with ssh command
<clever> o1lo01ol1o: it should be in PATH by default
<clever> o1lo01ol1o: what OS is the remote machine?
<clever> o1lo01ol1o: yeah
<clever> o1lo01ol1o: you need to mess with .bashrc and/or .bash_profile, to get nix-store in PATH for non-interactive shells
<clever> o1lo01ol1o: thats the difference between interactive and non-interactive shells
<clever> tobiasBora: home-manager is a bit different, its re-implementing (or copying) nixos modules, to get the same effect
<clever> that is one command, not 2
<clever> o1lo01ol1o: with the exact command i gave?
<clever> o1lo01ol1o: what happens when you `ssh user@host nix-store --version` ?
<clever> and then drop them somewhere
<clever> tobiasBora: if you eval a configuration.nix via <nixpkgs/nixos> you can then pull out the .unit files like this
<clever> nix-repl> config.systemd.user.units."pulseaudio.service".unit
<clever> «derivation /nix/store/xm1ll198x2hs2cfgg4b8bh9sp2d1v0an-unit-pulseaudio.service.drv»
<clever> tobiasBora: it can be done with some effort
<clever> tobiasBora: mostly a limitation of the nix expressions in nixpkgs
<clever> tobiasBora: systemd.user.service only works in configuration.nix
<clever> Henson: its much simpler when using nix copy
<clever> so less data has to copy if the /mnt device already has it
<clever> it has the benefit of reusing objects already in /mnt/
<clever> yeah
<clever> how you get the /mnt/ dir over is up to you, usb, nfs, ssh, carrier pigeon
<clever> 3/4 are done on the "target"
<clever> 1/2 are done on a "deployer" type machine
<clever> that wil make it active
<clever> Henson: step 4: nix-env --profile /nix/var/nix/profiles/system --set /nix/store/hash-something && /nix/store/hash-something/bin/switch-to-configuration switch
<clever> that will copy it back out of the usb stick, and into /nix/store on the host
<clever> Henson: step 3, nix copy --from local?root=/mnt/ /nix/store/hash-something
<clever> that will copy the closure of it into /mnt/nix/store, which can just be a USB stick
<clever> Henson: step 2, nix copy --to local?root=/mnt/ ./host1.root
<clever> that will build the given config and make a result root
<clever> Henson: step 1, nix-build '<nixpkgs/nixos>' -A system -I nixos-config=./host1.nix -o host1.root
<clever> Henson: ive looked into that before
<clever> colemickens: nix-diff

2019-06-06

<clever> and then it doesnt have enough free space to update the state
<clever> which requires a large change to the state
<clever> the problem is that its trying to delete everything at once
<clever> Avaq: keep running that, while increasing the number slowly
<clever> Avaq: try `nix-collect-garbage --max-freed 1`
<clever> Avaq: is /nix sharing the FS with something else?
<clever> ahh, but its failing due to the disk being too full
<clever> Avaq: nix-collect-garbage --max-freed 1, should delete .trash for you
<clever> zfnmxt: nixos-rebuild will update the bootloader config
<clever> try restarting it?
<clever> zfnmxt: if outputs things when building
<clever> zfnmxt: is it building something or just chewing cpu?
<clever> Henson: if you want to test a nixos module, put it in /etc/nixos/ and use imports = [ ./thatfile.nix ];
<clever> Henson: environment.etc and systemd.services, in your nixos module
<clever> mjrosenb: was trying to think of something thats on both
<clever> mjrosenb: not easily
<clever> youll need to list the deps yourself on line 4
<clever> mjrosenb: nix-env works differently
<clever> thats just 2 derivations, that builds one, then the other
<clever> mjrosenb: but nix-shell cant take the shell for 2 things at once and merge it
<clever> let me try something...
<clever> yeah, i can see the trouble
<clever> what about having one shell for ghcjs, and then another shell for x86+emacs ?
<clever> ah
<clever> mjrosenb: does ghci work on ghcjs?
<clever> mjrosenb: why does emacs need the x86 version of things?
<clever> ah
<clever> mjrosenb: have you thought of just doing most of the development on x86, and testing with ghcjs every now and then?
<clever> mixing 2 ghc's like that may be difficult
<clever> mjrosenb: the .env is meant to be used with nix-shell, and cant be put into buildEnv
<clever> o1lo01ol1o: hydra-queue-runner is the user that connects to build machines via ssh
<clever> that should also work
<clever> and then give it read on the github repo
<clever> i just run `ssh-keygen` with no args (and no passphrase) as the hydra-queue-runner user
<clever> dont really remember, it was ages ago
<clever> aanderse: i once got banned from #debian for mentioning i was running ubuntu, lol

2019-06-05

<clever> samueldr: unpackCmd is a special hook that is ran multiple times
<clever> yeah
<clever> samueldr: because pre and post have to run before and after your install code
<clever> o1lo01ol1o: hydra-queue-runner is always the one connecting to build slaves
<clever> o1lo01ol1o: deployment.keys can be used to copy the private to the hydra when deploying
<clever> o1lo01ol1o: resources.ec2KeyPairs cant copy the private to the hydra machine, so its of no use here
<clever> o1lo01ol1o: ive just manually created a keypair, and hard-coded its public into the nix
<clever> if you didnt accept it, the search cant find it
<clever> error: You must accept the Android Software Development Kit License Agreement at
<clever> by setting nixpkgs config option 'android_sdk.accept_license = true;'
<clever> nix-repl> androidenv.androidPkgs_9_0.androidsdk
<clever> nschoe: it depends on the status of the license
<clever> CapsLock: are you using the package name, or the attribute path?
<clever> CapsLock: where did you find the name android-sdk?
<clever> https://nixos.wiki/wiki/Special:RecentChanges last change was ~3 days ago
<clever> tobiasBora: it comes and goes
<clever> hooo: screenshot?
<clever> hooo: what error does it give when starting?
<clever> hooo: what exactly is "ruined" with the vm now?, how is it failing?
<clever> so its just "missing" and has to be build again
<clever> tobiasBora: its less that it knows to rebuild, and more that if you change anything, the hash changes, and then it cant find $out in /nix/store/
<clever> tobiasBora: the $out path is based on a hash of the .drv file, which includes every build-time input, and the recipe for building it
<clever> gchristensen: yeah
<clever> correct
<clever> it doesnt care which var you got the input from
<clever> its always a subset of all inputs
<clever> tobiasBora: it will serialize $out into a nar, and then just search that whole blob for every hash that was in the input closure
<clever> and generated when the build is finished
<clever> yeah, that metadata is stored in db.sqlite
<clever> tobiasBora: nix why-depends
<clever> pretty much
<clever> and then $src will be that storepath
<clever> tobiasBora: it will be copied to /nix/store/<hash>-myfile, based on the hash of its contents
<clever> tobiasBora: /nix/store/
<clever> it will just copy the file, without creating a derivation
<clever> $out/bin/.wolframscript-unwrapped
<clever> but you could borrow the naming scheme from it
<clever> it cant really run X on Y while copying Y
<clever> yeah, in this case, it would
<clever> tobiasBora: wrapProgram does that rename for you
<clever> i believe makeWrapper always passes all other args onward
<clever> wrapProgram is a helper, that just renames the binary, and then calls makeWrapper with the original and new name
<clever> ive made a wrapper for python, that adds a single "flag" to it
<clever> makeWrapper ${python}/bin/python $out --add-flags ${./exchange-monitor.py}
<clever> oh, now i do, one min
<clever> ive done similar, i cant remember which package though
<clever> colemickens: try parsing this!
<clever> nix run nixpkgs.yarn nixpkgs.haskellPackages.purescript --arg config '{ allowBroken = true; packageOverrides = pkgs: { haskellPackages = pkgs.haskellPackages.extend (self: super: { purescript = pkgs.haskell.lib.doJailbreak super.purescript; }); }; }'
<clever> mac10688: i just have freakish memory, ive memorized the entire nixos option set
<clever> mac10688: `nixos-rebuild switch` will apply all changes
<clever> mac10688: also, once it boots, you dont need to touch nixos-install
<clever> mac10688: did you read the xserver example in configuration.nix?
<clever> so you have no way to sign in
<clever> mac10688: and because of `--no-root-passwd`, you didnt set a root pw
<clever> mac10688: just mount the rootfs and bootfs to the same spot again, and nixos-install will update it
<clever> mac10688: it can resume an existing partially done install
<clever> o1lo01ol1o: hydra is configured to never build unfree stuff, because that would then be sharing it to people who didnt agree to the license
<clever> mac10688: yeah, /dev/nvme0n1p1 is your ESP, so you want to mount that to /mnt/boot/
<clever> o1lo01ol1o: why do you need uufree packages in release-lib.nix?
<clever> then ssh in from that second computer
<clever> mac10688: passwd root ; systemctl start sshd
<clever> can you pastebin the output of `fdisk -l` ?
<clever> you can mount the existing ESP at /mnt/boot/ then
<clever> did you make the efi system partition in fdisk?
<clever> filesystem
<clever> if your using efi, then it must be the efi system partition at /mnt/boot/
<clever> mac10688: did you mount an FS to /mnt/boot/ ?
<clever> mac10688: simplest thing is to give nixos full control of /boot/ and then use https://nixos.org/nixos/options.html#boot.loader.grub.extraentr to aextned the grub config

2019-06-04

<clever> simukis_: nix show-derivation /nix/store/foo
<clever> and more could likely be removed, but would depend on what your doing exactly
<clever> glenndavy: 80% of the things in your buildInputs wherent needed
<clever> glenndavy: can you pastebin the shell.nix file?
<clever> that will load . rather then <nixpkgs>
<clever> glenndavy: nix search -f . things
<clever> gchristensen: still need an answer? :P
<clever> 2016-01-15 21:11:11< gchristensen> henrytill: <turning a joking conversation serious> has there been any looking / work at an embedded nixos?
<clever> yeah, i beat you by about 6 months
<clever> 2016-01-15 20:56:56-!- gchristensen [~gchristen@unaffiliated/grahamc] has joined #nixos
<clever> --- Log opened Thu Jun 25 11:13:00 2015
<clever> $ head ~/irclogs/freenode/#nixos.log
<clever> gchristensen: dang!, i thought you where here before me!
<clever> the source shows signs of an ancient time when buildInputs wasnt a thing
<clever> at least since i started using nixpkgs (2+ years ago)
<clever> yes
<clever> glenndavy: the buildInputs should be added to PATH automatically
<clever> kraem: probably, i avoid systemd when possible
<clever> so if any fs has the type zfs, it will support zfs automatically
<clever> kraem: boot.supportedFilesystems is based on fileSystems.*.fsType
<clever> kraem: fdisk -l /dev/sda
<clever> kraem: yep, looks perfect
<clever> which is why i previously used lvm
<clever> kraem: it will remember the passphrase, and try it on the 2nd luks device for you
<clever> kraem: thats the recent improvement to nixos
<clever> but with recent improvements to nixos, you can just make 2 luks devices, one for zfs, one for swap, and share the passphrase
<clever> so zfs is being bypassed
<clever> kraem: my swap device on lvm on luks
<clever> kraem: what gchristensen said ^^
<clever> kraem: yeah, youll want to read that script as an example of how it works, and then run parts of it
<clever> inquisitiv3: close, callPackage also adds a .override function, to let you change the args later
<clever> kraem: this bash script will setup zfs on luks, and due to swap, it will also have lvm, but you could skip the lvm
<clever> and it will first search your set, then search pkgs
<clever> the trick here, is that you can callPackage = pkgs.newScope { a = 42; };
<clever> ah
<clever> > (pkgs.newScope { a = 42; }).callPackage ({ a }: a) {}
<clever> > (pkgs.newScope { a = 42; }).callPackage ({ a }: a)
<clever> newScope is also of use
<clever> tjay: obs and steam both work for me
<clever> infinee: services.openssh in configuration.nix
<clever> since the remote end has no way to ever unlock it
<clever> i dont think it would support that mode
<clever> hyper_ch: i suspect they share the master key then
<clever> so you cant just run a zfs command to expose it to the world
<clever> never revealed to the user
<clever> pie__: the master key wont change, and it may be possible to undo a pw change
<clever> so you may try to decrypt the wrong disk with that header, or accidentally format over it
<clever> so there is no way to reasonably locate it, or even tell if its there
<clever> but also, with a detached header, the luks partition wont even have a uuid
<clever> yeah, luks has a detached header option
<clever> and any of those passwords can unlock the master key
<clever> rather then just storing a single encrypt(master,pw), it stores a list of them
<clever> luks also takes advantage of that, to add more features
<clever> seems like such an obvious problem, and no real cheap way to solve it, same reason luks has this issue
<clever> but that doubles your usage
<clever> and then maybe send|recv between the 2
<clever> for zfs, you could just make a new dataset, which doesnt inherit the key from another dataset
<clever> but that involves re-writting all data to disk
<clever> the only solution is to change the master key at regular intervals
<clever> yep
<clever> pie__: when you change your passphrase, your basically just doing encrypt(decrypt(cipher,oldpw),newpw)
<clever> pie__: the real key used for encryption of the entire disk, is encrypted with the passphrase, and stored somewhere on-disk
<clever> and think that changing the passphrase is enough
<clever> you may not even know such a clone exists
<clever> and read any future data youve written
<clever> an attacker with that initial clone, can just undo the password change next time they gain access
<clever> ashkitten: and then 2 years more down the road (5 years after the clone), your password was leaked, and you immediately changed the phrase in zfs
<clever> ashkitten: the problem, is what if an evil maid cloned your drive 3 years ago
<clever> luks has the same issue
<clever> then an attacker can undo a password change
<clever> if you just protect the real key with a second passphrase, and your just re-encrypting it with a new passphrase
<clever> if you change the main key, then any future data is safe, but you have to re-encrypt every single block
<clever> ashkitten: thats the difference between changing the main key, vs wrapping the main key with a second passphrase
<clever> they have the old master block, and can effectively undo your password change
<clever> even if you change the password immediately
<clever> and if your passphrase is ever leaked in the future, and the attacker has a copy of the master-key, encrypted with that, he can then break it
<clever> if he also copied the (encrypted) master key block each time, and keeps an eye on leaked passwords
<clever> but, the attacker would only have the cihpertext, initiallly
<clever> reading the heckel.xyz blog
<clever> pie__: since send|recv can cpy encrypted datasets, an attacker with physical access can clone your dataset, and use incremental sends to update his clones if he regains access
<clever> pie__: and reveals something mildly scarry
<clever> pie__: the master key part is very much like luks
<clever> by default, it wont use more then 50%, and it can shrink down to as low as 32mb
<clever> ashkitten: `arcstat.py -v` explains all the other fields
<clever> for my system, its sitting at 1.1gig right now
<clever> ashkitten: c is how much ram zfs wants to use, arcsz is how much its actually using
<clever> arcstat.py -f time,read,dmis,dhit,dh%,mrug,mru,mfu,mfug,mread,c,ph%,pm%,mm%,miss,hits,mhit,mh%,mmis,hit%,eskip,arcsz 10
<clever> definitely dont put a pools vdev onto a volume within the same pool :P
<clever> pie__: no idea
<clever> filesystems and encryption are then built ontop of that, and dont decide where to write things, just "save this block", "where did you put it?"
<clever> if DDT is on, it does a lookup (based on hash), and may return an old location
<clever> if DDT is off, thats the location
<clever> you give it a block, and it returns a block-id
<clever> also, i think the DDT operates at a low level, where you just have a raw block storage