2019-09-28

<clever> boxscape: what pid/process do the <number>'s refer to?
<clever> boxscape: do you have a nix-shell open?
<clever> boxscape: try the --query as root?
<clever> boxscape: lol!
<clever> boxscape: you will need to depending on what {censored} is, you will need to delete it, but the safe way to do so differs depending on what it is
<clever> boxscape: `nix-store --query --roots <thatpath>`
<clever> boxscape: try running `nix-store --delete` on that path, and dont use the force flag
<clever> dsg: i prefer setting it to a specific rev, but you can also point it to a local channel (managed by nix-channel) or just directly to a channel
<clever> # nixops modify -d house deployments/house.nix -I nixpkgs=https://github.com/nixos/nixpkgs/archive/dae9cf6106d.tar.gz
<clever> 2019-09-28 11:48:09 < pie_> ah ok, but i just meant for the usersapce, like nix-shell -I nixpkgs=channel:nixos-unstable zfs or something
<clever> dsg: nixops modify -d name foo.nix -I nixpkgs=something
<clever> boxscape: that corrupted some files, try doing a `nix-collect-garbage` to see if it can delete the corrupt ones
<clever> boxscape: have you had any improper shutdowns lately?
<clever> jluttine: or the sudoers file?
<clever> jluttine: have you looked into systemd user services?
<clever> boxscape: yep
<clever> boxscape: nix-store --verify --check-contents
<clever> boxscape: cat /nix/store/7whxa62xjyfylxhww5w9ppjpsl4x9rim-stdenv-linux/setup
<clever> boxscape: what options is /nix mounted with? what does that setup file contain?
<clever> eraserhd: _modules.args is used to add args to all modules
<clever> pie_: that might help
<clever> pie_: modinfo shows the dependencies of a single module
<clever> 0.7.13-1
<clever> [root@amd-nixos:~]# cat /sys/module/spl/version
<clever> and `rmmod spl` removes it from `lsmod` ?
<clever> pie_: spl also reloaded?
<clever> its part of spl
<clever> then just `modprobe zfs`, and it will load whatever it needs
<clever> unload everything with a z in it!
<clever> pie_: does it say which module its in?
<clever> i cant find any trace of procfs_list_install is my local copy of the kernel source
<clever> pie_: is procfs_list_install spelled correctly?
<clever> pie_: nixos-rebuild cant reload kernel modules
<clever> or use -I on nixos-rebuild?
<clever> pie_: did you mess with the channels before the nixos-rebuild?
<clever> then you also need to reload spl
<clever> name another!
<clever> pie_: name one
<clever> pie_: is spl in `lsmod` ?
<clever> pie_: what unknown symbol? what does dmesg say?
<clever> 2019-09-28 11:44:07 < clever> pie_: try turning on enableunstable then, from the iso, then `rmmod zfs` and `modprobe zfs` to reload it, dont update the nix channels
<clever> pie_: thats why i said to rmmod first, a few mins ago
<clever> pie_: correct, modinfo looks at the fs, not the live kernel
<clever> pie_: did you `rmmod zfs` first?
<clever> pie_: realpath $(which zfs)
<clever> pie_: zfs --version?
<clever> pie_: the kernel part is the most important one i think
<clever> ,pills
<clever> pie_: which is why i said to not update channels
<clever> pie_: if you mess with the channel, the zfs module wont fit your kernel version, and you cant load it
<clever> pie_: userspace is the part that will update the easiest
<clever> xok: ive never used that
<clever> pie_: what did it fail with?
<clever> pie_: try turning on enableunstable then, from the iso, then `rmmod zfs` and `modprobe zfs` to reload it, dont update the nix channels
<clever> pie_: i typically travel with 3 or 4 random usb sticks, lol
<clever> xok: you can use `nixops set-args --arg foo 42` to pass an argument to the deployment file, and then use `lib.range` and `lib.map` to dynamically generate N machines
<clever> pie_: do you have 2 usb sticks?
<clever> but you cant reboot the iso
<clever> pie_: changing zfs flags at runtime is tricky, and its best to reboot first
<clever> pie_: did you set any zfs options in configuration.nix?
<clever> pie_: nix-du does that
<clever> that removes a root, and then normal gc can delete it
<clever> removing generation 500
<clever> [root@amd-nixos:~]# nix-env --profile /nix/var/nix/profiles/system --delete-generations 500
<clever> [root@amd-nixos:~]# nix-env --profile /nix/var/nix/profiles/system --list-generations
<clever> pie_: nix-env --delete-generations
<clever> and then grub has to go thru /nix/store/ to find things
<clever> if they are on the same partition, nix wont copy kernels automatically
<clever> ah
<clever> though i prefer to not -d until ive confirmed it can still boot, and i generally delete generations selectively
<clever> 11:36:58 up 24 days, 14:33, 2 users, load average: 0.13, 0.18, 0.18
<clever> pie_: the system generations wont let you gc rollbacks (enless you -d)
<clever> tilpner: and debugging it would be imposible when it does break
<clever> tilpner: so i have no idea how it works
<clever> tilpner: ive tested in a vm, and it can boot with the default flags, but EVERY attempt to list directories with grub fails
<clever> pie_: do you know if /boot and /nix are on the same zfs filesystem/dataset?
<clever> which is why i dont like /boot on zfs
<clever> pie_: its more likely that zfs just upgraded, and grub doesnt support the new zfs flags
<clever> pie_: and the nixos gc roots, wont allow you to gc something grub can still refer to, the GC'ing wont have broken it
<clever> pie_: just reboot it
<clever> if you can import read-only, then you can read the configuration.nix to confirm
<clever> what error does `zpool import foo` give?
<clever> thats fairly recent then, i guess your next option is to get the zfs mounting
<clever> so it cant be GC'd
<clever> pie_: my recovery boot module, copies the kernel to /boot/
<clever> pie_: what is the last-mod timestamp on that?
<clever> pie_: what path is it at, within the partition
<clever> Henson: use pkgs.pkgsi686Linux.buildFHSUserEnv
<clever> partitions exist outside of zfs
<clever> pie_: you dont have to load the zfs to check for other partitions
<clever> pie_: can you boot any nixos iso, and pastebin the `fdisk -l /dev/sdsomething` ?
<clever> pie_: you dont have a cellphone?
<clever> pie_: can you screenshot the exact error?
<clever> pie_: then its always copying kernels, and ignores the copyKernels flag
<clever> pie_: do you have /boot on its own filesystem?
<clever> ivan: nixos-install is basically just a wrapper that runs nixos-rebuild in a chroot
<clever> and the rest of the steps are just normal nixos install stuff
<clever> ivan: step 3 is just a slight tweak to the `nix copy` morph says its using
<clever> ivan: also, i have an open issue on nixops, that would do what you said above
<clever> ivan: sounds like nixops
<clever> ivan: morph?

2019-09-27

<clever> samueldr: what happens fi you run `nix-collect-garbage --max-freed 1` ?
<clever> gchristensen: that file also has derivations for signing things, but the keys wind up in the nix store
<clever> 67 locks requirements for signatures on, then 69/70 fetch a script/sig, and validate it, then execute it
<clever> gchristensen: but with the script= on line 65, it will embed that into the binary, and always run that on bootup, so it wont dhcp, and wont run whatever dhcp tells it to
<clever> gchristensen: the embeded script is also of use, without that, ipxe will default to not needing signatures
<clever> gchristensen: dummy certs/keys are in the repo (for CI) along with a regen script, because they keep expiring (and breaking CI)
<clever> gchristensen: yep, its in not-os
<clever> __monty__: but if the history is long, it can be slower
<clever> __monty__: if things are changing often, it can use `git fetch` to only fetch the diff
<clever> yep
<clever> __monty__: builtins.fetchGit or builtins.fetchTarball can do that
<clever> __monty__: the eval-time fetchers dont need a hash, but slow the eval down
<clever> pretty much :P
<clever> because if it didnt, using the wrong callPackage will result in things not obeying overlays
<clever> exarkun: i think callPackage has been specially crafted to always pull from self
<clever> exarkun: i would just put that into the overlay file
<clever> exarkun: oh wait, why does twisted need twisted??
<clever> exarkun: use self.callPackage, not super.callPackage
<clever> alexarice[m]: ive only gotten it to work with blueman, which is graphical
<clever> bluetooth is like that
<clever> alexarice[m]: yep, and that looks good
<clever> alexarice[m]: what does this output?
<clever> ls -l /run/current-system ; realpath /nix/var/nix/profiles/system
<clever> alexarice[m]: have you rebooted since it was last working?
<clever> exarkun: just saying that it does work on nixos
<clever> alexarice[m]: run pavucontrol, do you see the bluetooth device? has pulseaudio been restarted? `pactl exit`
<clever> exarkun: ive also been able to make the laptop impersonate a headset, and connect to the cellphone, so audio came out of the laptop
<clever> exarkun: ive gotten bluetooth headsets to work using blueman on both my desktop (usb dongle) and laptop (builtin)
<clever> inkbottle: i use blueman to manage bluetooth
<clever> exarkun: thats required to get bluetooth support in pulse
<clever> dminuoso: that should work, but i also cant find any such examples
<clever> dminuoso: this one checks that if withFonts is true, freetype must not be null
<clever> /home/clever/apps/nixpkgs-master/pkgs/development/libraries/libbluray/default.nix:assert withFonts -> freetype != null;
<clever> dminuoso: asserts can do that

2019-09-26

<clever> and your likely to want initrd ssh, when using nixops, since the hardware is remote
<clever> mrSpec: deployment.keys will copy keys to that dir
<clever> mrSpec: i think thats a nixops specific path
<clever> mrSpec: you may need a `nix-collect-garbage -d` to blow them away, or more selective GC'ing
<clever> mrSpec: i think the path is baked into older generations, and as it generates the initrd's for the old ones, it looks at the old path
<clever> mrSpec: put the key somewhere that will persist, /run is wiped on reboot
<clever> exarkun: but func = a: func a; wont
<clever> exarkun: so a = a; will fail
<clever> exarkun: infinite recursion is detected if you try to eval another thunk that is in your current call-stack
<clever> exarkun: something like that
<clever> exarkun: foo.override (old: { bar = old.bar ++ [ "baz" ]; })
<clever> exarkun: .override can take a function
<clever> pie_: your paste with the example expired!
<clever> exarkun: checking the logs...
<clever> exarkun: i would just merge the 2 overlays into one, simplest option
<clever> gloaming: what did it say?
<clever> gloaming: or journalctl -f -u display-manager.service -n100
<clever> niso: so the build nixops has made, can be used directly as the 1st generation
<clever> niso: more about replacing the nixos-install step, with a nix copy step
<clever> was thinking more about the code that runs nixpart, then installs nixos
<clever> niso: is the shell script that runs it, within the nixops repo?
<clever> niso: hmmm, the only script i can find is for digitial ocean, not hetzner
<clever> niso: i think that is currently using regular nixos-install, so its still creating a dummy generation you cant really use?
<clever> niso: have you seen my nixops issue that might be able to make use of nixpart features?
<clever> l33: yes
<clever> exarkun: super, is everything up to the one before current
<clever> exarkun: self is the final version, after applying all overlays, including future ones in the list
<clever> inkbottle: which part are you stuck at?
<clever> inkbottle: ctrl+alt+f1, login as root, with the pw you set
<clever> inkbottle: you will also want to umount things before the reboot
<clever> inkbottle: after you boot the machine, you can login as root, and use passwd to change other user pw's
<clever> inkbottle: thats normal
<clever> exarkun: i think you need to use lib.composeExtensions to merge 2 python overlays into one python overlay, then use that
<clever> inkbottle: it will only merge if those entries are in different files, and one loads the other thru imports
<clever> exarkun: super.python27.override overwrites the packageOverrides, it doesnt append to it
<clever> inkbottle: the contents between the { and } of one, must be moved into the {} of the other
<clever> inkbottle: same problem, you still have 2 lines that start with `services.xserver = {`, they must be merged together
<clever> dminuoso: ive seen some people do, { pkgs ? import <nixpkgs> {}, hello ? pkgs.hello }:
<clever> inkbottle: i would need to see the new version of the config file
<clever> ivan: ah, external postgres server? ive never tried that mode
<clever> ivan: journalctl -f -u hydra-init.service ?
<clever> ivan: what error did it have in that service?
<clever> inkbottle: services.xserver is present twice in the file, that wont parse, youll need to merge the 2 of them together
<clever> ivan: does the journal for it say anything when hung?

2019-09-25

<clever> iqubic: `nix show-config | grep max-jobs` ?
<clever> hodapp: try the 2nd one then, with --store
<clever> hodapp: yep
<clever> hodapp: or `nix-store --store local?root=/mnt --delete /nix/store/foo`
<clever> hodapp: nixos-enter
<clever> hodapp: run `nix-store --delete` on the storepath, what does it say?
<clever> hodapp: and nixos-rebuild is just a helper script that runs nix-build
<clever> hodapp: nixos-install is basically just a helper script, that runs nixos-rebuild in a chroot
<clever> since ext4 was interupted mid-write
<clever> hodapp: could have similar effects though
<clever> hodapp: nix-store --verify --check-contents, will validate everything
<clever> hodapp: but ext4 can commit things out of order, and i have seen files get truncated after nix flagged them as valid, after an improper shutdown
<clever> hodapp: nix wont register a path as valid in db.sqlite until it has finished writing it
<clever> exarkun: that leads to you spending 6 hours trying to get your changes to print anything at all :P
<clever> exarkun: if you then `nix-build -A nixops` and ./result/bin/nixops, it will use the nixops from nix-shell, not from result
<clever> exarkun: ive run into related problems, if you `nix-shell -p nixops`, it will leak a PYTHONHOME
<clever> gchristensen: i'm using buildLayeredImage and am winding up with 3gig layers, got any tips on how to debug it, or should i just increase the layer count more?
<clever> spazzpp2: what about the dmesg output?
<clever> spazzpp2: that would be the problem, it shouldnt be empty
<clever> spazzpp2: what is the contents of that file?
<clever> EFI eliminates that issue
<clever> but grub also almost never updates
<clever> ive yet to break a machine, lol
<clever> then the MBR just never updates
<clever> personally, i usually just set it to "nodev" after i install
<clever> so it can still boot, but you will likely want to fix it, or you risk overwriting the MBR of the wrong disk
<clever> hodapp: which only happens when the grub version changes
<clever> hodapp: nixos will only use that device setting when it has to re-install the bootloader to the MBR
<clever> sphalerite: so you can either write a dhclient hook, or switch to dhcpcd, and use a dhcpcd hook
<clever> On after defining the make_resolv_conf function, the client script checks for the presence of an executable /etc/dhcp/dhclient-enter-hooks script
<clever> Type: one of "dhclient", "dhcpcd", "internal"
<clever> networking.networkmanager.dhcp
<clever> sphalerite: does network-manager have any dhcp hooks? what dhcp client does it use?
<clever> and i then echo'd them to a file, in the right format, to include into the bind cfg
<clever> sphalerite: the dns servers are in an env var when that hook gets ran
<clever> Shell code that will be run after all other hooks. See `man dhcpcd-run-hooks` for details on what is possible.
<clever> networking.dhcpcd.runHook
<clever> sphalerite: `man dhcpcd-run-hooks`
<clever> sphalerite: `Hooking into events` in `man dhcpcd`, it will run some shell scripts when it does things
<clever> sphalerite: ah right, i did that on an old gentoo machine, let me see what the exact method was...
<clever> sphalerite: ive done that before, let me see
<clever> slabity: `nix show-config` also shows the result of parsing the config
<clever> slabity: --option just lets you override any nix.conf entry
<clever> gchristensen: packet.net could be a real pxe boot, with the installer image
<clever> gchristensen: aws would just have an ami, that is basically the installer iso with ssh enabled, to let you in
<clever> gchristensen: and the kexec is an optional function, that some backends can choose to use, but the backend must supply a way of getting root on a rescue shell or ubuntu image
<clever> gchristensen: my idea, is that steps 2/3/4 are all a common library, that any nixops backend can use
<clever> then you just reboot, and its live!
<clever> which installs the bootloader for an already copied nixos
<clever> step 4, nixos-install --system /nix/store/hash-nixos --root /mnt
<clever> which copies to /mnt/nix/store on a remote box
<clever> step 3, nix copy --to ssh://root@target?remote-store=local?root=/mnt /nix/store/hash-nixos
<clever> step 2 is to partition, format, and mount everything under /mnt
<clever> pxe, kexec, an ami that contains the netboot image by itself
<clever> gchristensen: step 1 is to somehow boot nixos without installing it
<clever> gchristensen: my nixops plans only trigger once, at install time
<clever> and also, the binary cache not working on the initial deploy
<clever> and bypasses having to reboot after the first deploy because wireguard cant modprobe
<clever> it also bypasses needing a generation 1 that wont match your nixpkgs and has to be GC'd when your done
<clever> that will let nixops support custom partitioning and FS types, on any backend
<clever> infinisil, gchristensen: did you see my nixops issue, that could use nixpart features?
<clever> hodapp: because then changes you make to that stick persist, and you can leave a clone of your config repos, and ssh priv keys on it
<clever> hodapp: nixos-install works on any nixos machine, and i do prefer the idea of just installing to a usb stick
<clever> and then you can do whatever you want
<clever> the reason i was thinking haskell and rpc, is that the guest could create any kind of object, serialize it, and fire it off to the host, which deserializes
<clever> and also stream logs over, in real-time
<clever> exarkun: the guest side can then do lines 34-55, and spit out a json blob? with status
<clever> exarkun: step 1, boot the vm, step 2, tell the guest half of the code to do everything else
<clever> exarkun: for this test for example, i just want the host side to be a 2 step process
<clever> its just a single raw byte stream
<clever> network and tcp isnt going to be available
<clever> does haskell have anything close to that?
<clever> exarkun: the issue, is that i basically need an RPC channel, that can act over a byte stream, and is mux-able (can deal with parallel concurrent streams)
<clever> exarkun: and then haskell code in each guest, doing actions, and reporting status over the rpc
<clever> exarkun: so you can have haskell code in the host, acting as a conductor
<clever> exarkun: with haskell code on both the client and server side (host and guest)
<clever> exarkun: what i want, is a haskell library, with a muxed RPC over that serial console
<clever> exarkun: that is something i have wanted to solve
<clever> exarkun: also behind the scenes, it uses a qemu hypervisor serial port, to get a backdoor shell into the guest
<clever> exarkun: behind the scenes, all things in the guest, are ran with ( foo ) ; echo '|!EOF' $?
<clever> exarkun: of note, is how succeed even gets exit codes
<clever> exarkun: yep, and its bi-directional
<clever> exarkun: 63 then copies it to $out
<clever> exarkun: line 41 writes a video file to /tmp/xchg/ in the guest
<clever> NemesisD: nix-env -f https://sometjing -iA bar
<clever> |systemd-cat would let it leak into the logs, which are saved
<clever> succeed may return the stdout as a perl variable
<clever> try that?
<clever> literally, print "foobar";
<clever> exarkun: `print`
<clever> exarkun: if you want binary files, xchg is an option
<clever> exarkun: ah, you can only write to it from outside the qemu
<clever> exarkun: and systemd-cat
<clever> exarkun: have you seen the xchg stuff?
<clever> exarkun: when exactly are you trying to edit it?
<clever> exarkun: have you seen the index.html the tests generate?
<clever> sounds right
<clever> exarkun: yep, so that is the one thing you can write to that will persist
<clever> exarkun: how does $out and $LOGFILE compare, for the same drv?
<clever> exarkun: what is the value of $out ?
<clever> tetdim: now its bothering me, what you censored out, lol
<clever> tetdim: and you must somehow tell meson that local-store.cc depends on schema.sql.gen.hh
<clever> tetdim: it must be in a directory that can be found with the -I flags while building local-store.cc
<clever> so you know what make is doing
<clever> tetdim: all trace-gen does, is print `GEN schema.sql.gen.hh`
<clever> tetdim: so your thing with cat should be fine
<clever> tetdim: yep, thats a trace, that prints GEN
<clever> mk/tracing.mk: trace-gen = @echo " GEN " $@;
<clever> not to mutate the stream