2018-09-07

<clever> iclanzan: is /boot mounted?
<clever> infinisil: the cross-compile changes also sorta broke the non-cross compile modes in nixos, when you just want to force a 32bit nixos build
<clever> not sure, i havent re-memorized all of the core files after the cross-compile changes went thru
<clever> inherit (config.nixpkgs.localSystem) system;
<clever> gchristensen: i think you still want to set system, when importing the 2nd nixpkgs
<clever> gchristensen: nixpkgs.localSystem.system is a nixos option, so it has to be set inside myfile.nix or inside the configuration set
<clever> gchristensen: oh, and i believe this was a change on the nixos side, not the args to <nixpkgs>
<clever> gchristensen: the cross-compile stuff changes the syntax a bit
<clever> gchristensen: have you tried nixpkgs.localSystem.system = "i686-linux"; ?
<clever> xok: also check `tail -F /var/log/zabbix/zabbix_server`
<clever> any errors?
<clever> xok: check the journal for the service, journalctl -f -u zabbix-server.service
<clever> xok: sure
<clever> which backend is nixops using? where is the vm hosted?
<clever> xok: did you open port 10051 in the firewall?
<clever> xok: what does `netstat -anp | grep zabbix_server` report?
<clever> that option will create another systemd service
<clever> xok: ah, then you probably want services.zabbixServer.enable = true;
<clever> xok: by default, it only listens on 127.0.0.1, so you cant access it from outside the machine
<clever> xok: `systemctl status zabbix-agent.service` should say its running, when you are ssh'd into the machine
<clever> xok: it will also add the agent to the system PATH
<clever> xok: when enable is set, it will auto-create a zabbix user in /etc/passwd, and a systemd service called zabbix-agent
<clever> yorick: the disk image, that will become the AMI, was too small for the install
<clever> yorick: line 1136, "No space left on device" is happening inside the vm, so yeah
<clever> yorick: oh!
<clever> yorick: can you pastebin the output of `df -h` and the full output from nix-build?
<clever> yorick: what about `df -i /` ?
<clever> ah, but store is still kk
<clever> df -h / /nix/store /tmp
<clever> oops, typo
<clever> yorick: does this command show the same line 3 times? `df -t / /nix/store /tmp`
<clever> yorick: is this on nixos?
<clever> i think it uses 2 or 3gig, not sure though
<clever> yorick: run `watch df -h` while the build is running, and see which volume is loosing free space
<clever> srhb: ah, i think its only used bu builtins.fetchurl, and the builtin:fetchurl builder
<clever> srhb: and fixed-output derivations will also check tarballs.nixos.org
<clever> legacy on gpt requires a dedicated bios boot partition, 1mb, not mounted, not formated
<clever> if you wanted to
<clever> but you can do both legacy and efi at the same time
<clever> when doing efi, it must be set to "nodev"
<clever> boot.loader.grub.device is only for legacy
<clever> sophiag: what error give grub give?
<clever> grub also has a lot more config options
<clever> so if your on an older machine, systemd-boot wont work
<clever> of note, systemd-boot is uefi only
<clever> ive been a grub person since grub 1
<clever> it has its own menu
<clever> systemd-boot doesnt have anything to do with grub
<clever> nixos-install is a script to run nixos-rebuild in a chroot, basically
<clever> sophiag: yeah
<clever> why did you mount something to / ?
<clever> sophiag: its using the config on /mnt when installing to /mnt
<clever> callPackage ({stdenv }: stdenv.mkDerivation { ... }) {}
<clever> so you can do:
<clever> and the {} at the end is overrides
<clever> then if the first arg is a function with args, it will query the names of the args it accepts, and auto-call it with those args from pkgs
<clever> ldlework: if the first arg is a file, it will import it
<clever> derped: Bisecting: 137 revisions left to test after this (roughly 7 steps)
<clever> derped: Bisecting: 278 revisions left to test after this (roughly 8 steps)
<clever> derped: it was broken about 3 months ago
<clever> you would need to import the file directly, and not use an overlay
<clever> yeah
<clever> then either import its file directly, or add them to pkgs?
<clever> your probably better off just editing nixpkgs directly, and maybe filing a PR
<clever> ldlework: ive tried pointing <nixpkgs> to a default.nix before, and it causes weird problems, because of the fact that you can <nixpkgs/lib/options.nix>
<clever> *looks up*
<clever> my laptop has 512mb, and ran out due to rescue_boot, lol
<clever> it also recently got full, and i had to disable my rescue_boot
<clever> most of it is kernels and initrds
<clever> 66M /boot/kernels
<clever> 9.6M /boot/grub
<clever> sophiag: my /boot on the laptop is using 79mb right now
<clever> yeah, thats why you need the boot.loader.efi.efiSysMountPoint = "/boot/EFI"; option in nixos
<clever> ah
<clever> emily: the kernel would be in /boot or /nix/store, not /boot/efi
<clever> sophiag: i think you could get away with 1mb, lol
<clever> -rwxr-xr-x 1 root root 119K Oct 29 2017 /boot/EFI/BOOT/BOOTX64.EFI
<clever> [root@system76:~]# ls -ltrh /boot/EFI/BOOT/BOOTX64.EFI
<clever> if your using ext4 for /, then you can put the ESP at /boot/efi and just omit the /boot partition entirely
<clever> uefi shouldnt have issues like that
<clever> sophiag: that problem is also part of why you may find /boot on older legacy machines, the bios cant read the entire root fs
<clever> sophiag: thats more about the start of the boot partition on the disk, and its only an issue for legacy booting
<clever> then you are free to make /boot ext4, or even use the /boot directory of /
<clever> but, you can also mount the vfat to /boot/efi, and set boot.loader.efi.efiSysMountPoint = "/boot/efi";
<clever> when doing efi on nixos, the default is a vfat mounted to /boot and /boot/efi is just a normal directory
<clever> sophiag: yeah
<clever> in my case, there is traces of the original OS left behind
<clever> Boot0004* UEFI OS HD(1,GPT,27c99b08-455d-4dfe-a44f-6150cbc09ef8,0x800,0x100000)/File(\EFI\BOOT\BOOTX64.EFI)..BO
<clever> Boot0000* ubuntu VenHw(99e275e7-75a0-4b37-a2e6-c5385e6c00cb)
<clever> the above efibootmgr command will print the current efi vars
<clever> so if you boot a USB stick in legacy mode, it cant setup the efi vars
<clever> sophiag: canTouchEfiVariables only works if you have booted via uefi
<clever> [root@system76:~]# nix run nixpkgs.efibootmgr -c efibootmgr -v
<clever> ah
<clever> then you are booting via uefi
<clever> sophiag: if you run `mount`, do you see this being mounted?
<clever> efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
<clever> v0latil3_: yeah, but then you need to deal with managing secrets in /nix/store/
<clever> i also didnt bother setting a bios password, so you can just turn secure-boot back off
<clever> just a hash of the grub binary
<clever> but grub will happily run anything you point it at :P
<clever> so my laptop can only ever run the grub.efi file nixos installed, and nothing else
<clever> v0latil3_: however, the grub binary itself, doesnt have any security enabled, and will happily run unsigned things, so it doesnt really protect you much
<clever> which just adds the hash of the binary to the whitelist
<clever> v0latil3_: i recently experiemnted with my laptop bios, and you can "enroll" any unsigned efi binary
<clever> sophiag: one of my haskell programs leaks memory at a fairly low rate, but after a few weeks or uptime, it has leaked 64gig or more :P
<clever> Henson: /tmp + "/foo"
<clever> v0latil3_: my desktop has 32gig of ram and 32gig of swap, and i soemtimes have to enable an extra 64gig swap parittion
<clever> https://github.com/cleverca22/nixos-configs/blob/master/rescue_boot.nix eats up about 300mb on /boot, and i tend to enable this on most machines
<clever> sophiag: i tend to make my boot partitions either 512mb or 1gig
<clever> sophiag: how big is it?
<clever> ldlework: overlays
<clever> "it boots windows, ship it"
<clever> sophiag: but some bios are so stupid, they even ignore the vars, and boot the filename windows puts the bootloader at :P
<clever> sophiag: with some bios firmware, you can manually edit those vars in the UI, which can help
<clever> booting from a uefi USB stick is one way to bootstrap that whole mess
<clever> sophiag: and if you move the hdd to another box, it wont boot, because the vars are missing
<clever> sophiag: so, you must boot via uefi, to configure it to boot via uefi, have fun! :P
<clever> sophiag: and you can only set those vars if you boot via uefi
<clever> sophiag: about the only thing i can think of, is that booting uefi generally requires some special vars being set in the firmware
<clever> sophiag: uefi booting required an efi system partition, with vfat, mounted to /boot (or maybe /boot/efi with the right config)

2018-09-06

<clever> which is just obeying the --prefix passed to configure
<clever> Henson: its usually generated by the makefiles of the project
<clever> and repeat for each path that fails to download
<clever> rsa: you will need to download that file manually, and then run `nix-prefetch-url file:///path/to/patch` to import it
<clever> rsa: why does it not have internet?
<clever> rsa: you could try adding `--option substituters ''` to disable it contacting anything
<clever> and if you updated the nixpkgs, they cant be reloaded, because the kernel versions would conflict
<clever> kernel modules arent reloaded when you rebuild-switch
<clever> yeah
<clever> then it will natively have unstable on every boot
<clever> tobiasBora: though for the netboot image i gave you, you can also set boot.zfs.enableUnstable = true directly in the nix expression that builds the initrd files
<clever> normally, you would just reboot to apply that updat,e but the livecd resets to defaults then
<clever> tobiasBora: rmmod all zfs and spl ones, then reload them with modprobe
<clever> tobiasBora: you need to reload the kernel modules
<clever> das_j: here is a different project i have, that builds against master and unstable, https://hydra.angeldsis.com/project/not-os
<clever> tobiasBora: nice
<clever> das_j: in my case, hydra is pre-building the whole thing against the latest nixos-unstable, so i can see if the new version breaks things before i try to update
<clever> das_j: yeah
<clever> das_j: you can use `--option substituters '....'` to override it temporarily
<clever> das_j: yeah, there is a bug in nix, that it doesnt give up if a cache fails to respond
<clever> das_j: run `curl cache.nixos.org/nix-cache-info`
<clever> das_j: if all of them are online, it will query them to see who has what, and then use the priority in the nix-cache-info file to pick one
<clever> tobiasBora: this is how the luks config is setup, to handle what srhb mentioned: https://github.com/cleverca22/nix-tests/blob/master/kexec/justdoit.nix#L123-L125
<clever> tobiasBora: i use zfs on luks without any trouble
<clever> boot.zfs.enableUnstable
<clever> tobiasBora: you may need to tell nixos to use zfs unstable
<clever> it means more ginea pigs for the testing :P
<clever> tobiasBora: also check the topic in #zfsonlinux , "Native encryption is not production ready, keep backups (but it works great)"
<clever> tobiasBora: your zfs version doesnt have encryption support
<clever> the above conversion util will show Y and Z are actually the same hash
<clever> it says hash was X, when Y was expected, but the nix file says Z
<clever> all output from nix uses base32, even if the input was base16, which can sometimes be a bit confusing
<clever> nix1 and nix2 support both types
<clever> srhb: you can use --to-base16 and --to-base32 to convert them
<clever> 7fb73b4cc6cc69d806d8025feba05dd827a13a2a386fa7f6d06288cb707ec1e1
<clever> [clever@amd-nixos:~]$ nix-hash --type sha256 --to-base16 1qf1grqcp232s3vafvrq58xa29yqbnhfnpq2v03dhsfcqr63pdvz
<clever> srhb: you may be comparing base16 and base32 hashes
<clever> layus: fonts.fonts = [ pkgs.carlito ];
<clever> ixxie: the laptop is where i run nixops, and that isnt part of the deployment
<clever> ixxie: yeah, the file i linked above manages my NAS and router
<clever> ixxie: by default, it just uses whatever <nixpkgs> maps to in $NIX_PATH
<clever> ixxie: but you can use `nixops modify -I nixpkgs=/path/to/something` to override that
<clever> on my laptop, that does a left drag
<clever> ixxie: also, try a quick tap, followed by a press and hold
<clever> also, try moving with 2 fingers
<clever> ixxie: now try with 3 fingers
<clever> ixxie: what happens if you tap with 2 fingers at once?
<clever> but you will loose the ability to see its output
<clever> yorick: the ssh dying will almost never kill the program within, linux retries the tcp send a lot, and can buffer a few mb of output
<clever> in my case, i'm using the toxvpn IP for some machines, so the IP is static, even if it changes to another interface
<clever> ixxie: the activate may finish on its own, but you will never see its output
<clever> ixxie: the wifi is more likely to change the IP, which then breaks all current ssh sessions
<clever> this can happen any time the nixpkgs rev changes
<clever> ixxie: see if ssh works now, and try re-running deploy
<clever> gchristensen: its also too late for the 18.09 branch, but we may want to cherry-pick it over, since the client refuses to connect to the older server
<clever> gchristensen: have you seen https://github.com/NixOS/nixpkgs/pull/46134 ?
<clever> Shell: you could also do systemPackages = [ pkgs.krb5Full.out pkgs.krb5Full.dev ]; in configuration.nix, to install both halves system-wide
<clever> nix-shell gives dev only binaries, nix-env doesnt
<clever> Shell: several of its binaries are in the .dev output
<clever> error: attribute 'krb5full' in selection path 'krb5full.out' not found
<clever> [clever@amd-nixos:~/nixpkgs]$ nix-build -A krb5full.out
<clever> > krb5Full.outputs
<clever> Shell: what does `nix-instantiate --find-file nixpkgs` return?
<clever> ixxie: the locks nixops uses are in the kernel, and the kernel will clean them up automatically upon the death of a process
<clever> ixxie: double check `ps aux | grep nixops` to confirm none are still running
<clever> ixxie: is there another nixops process still running?
<clever> a bug or PR will need to be filed on nixpkgs, and you can `nix-channel --rollback` to switch to an older nixpkgs until its fixed
<clever> Shell: a systemd unit would have been a much better place for that change
<clever> Shell: thats why i avoid activate scripts at all costs, one small mistake and the machine ceases to boot entirely
<clever> Shell: can you use this to diff the activate scripts, between the 2 generates, that work and fail
<clever> [clever@amd-nixos:~]$ diff -u --color /nix/var/nix/profiles/system-{415,416}-link/activate
<clever> Shell: what does `nix-channel --list` say?
<clever> Shell: did you do anything with activation scripts?
<clever> ixxie: nixops modify deployment.nix -I nixpkgs=/path/to/nixpkgs
<clever> that is how i got the listing
<clever> ls -l /nix/var/nix/profiles/system*
<clever> Shell: in this case, i switched from 18.03 to 18.09 (technically, i stayed on nixos-unstable, but the version of it has changed)
<clever> lrwxrwxrwx 1 root root 93 Jul 23 12:26 /nix/var/nix/profiles/system-420-link -> /nix/store/vkbs0vdaczlfscxz8dvjarczw8hikzwb-nixos-system-amd-nixos-18.09pre145679.dae9cf6106d
<clever> lrwxrwxrwx 1 root root 84 Jul 21 15:57 /nix/var/nix/profiles/system-418-link -> /nix/store/b5nqnf9l54254mx77zzcqdgp7x9glg3r-nixos-system-amd-nixos-18.03.git.d6c6c7f
<clever> Shell: start by comparing the git revs at the end of the names
<clever> tobiasBora: id need more time to study that before i can see what its doing
<clever> Shell: and what does `nix-channel --list` report?
<clever> Shell: and then run fsck to see if there is anything wrong with your disk
<clever> Shell: are you able to boot an older generate by selecting one at grub?
<clever> tobiasBora: and its less likely that you will break that 2nd nixos, so you can use it as a rescue system
<clever> tobiasBora: rescue_boot.nix is just putting the "netboot" files into /boot and adding a grub option, so you basically have a 2nd copy of nixos in /boot
<clever> and if you get it wrong, the machine will either forget how to boot, or systemd will drop you into rescue mode immediately, and wait for local input
<clever> you need to manually copy it to the nixops machine, and include it in the deployment files
<clever> ixxie: for which backend?
<clever> probably not, but it would likely need to me heavily modified to work with the stage-1 in nixos
<clever> yeah
<clever> it may also take longer to boot, since it has to copy the entire thing to ram
<clever> tobiasBora: so it will need more ram to boot, but once booted, you can just unplug the usb stick your booting from
<clever> tobiasBora: the major difference between an ISO and netboot, is that the netboot copies the entire (compressed) rootfs into ram
<clever> <nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-graphical-kde.nix> i think
<clever> tobiasBora: yeah, just add another thing to the imports field
<clever> tobiasBora: and it has many of the defaults an installer has, like auto-login for root
<clever> tobiasBora: the netboot stuff is a variant of the ISO, that puts the rootfs into the initrd
<clever> tobiasBora: you can even test the above from the live usb
<clever> tobiasBora: try running nix-build against that file now
<clever> pushed again
<clever> wait, that made it worse, lol
<clever> fix pushed
<clever> oh, but the filenames are off a tad
<clever> tobiasBora: run `nix-build multi-boot-helper.nix` and you will get a result symlink containing a kernel, initrd, and grub config file
<clever> i'm testing a modification here
<clever> so it wont really work for your idea of having an ISO that can do many things
<clever> rescue_boot.nix is a nixos module, that adds to the existing /boot of a nixos install
<clever> tobiasBora: testing an example locally
<clever> everything would be included in the initrd, including the squashfs
<clever> tobiasBora: let me type up an example...
<clever> you could then modify this a bit, to just get a kernel-params file, kernel, and initrd
<clever> and then just loading that with grub
<clever> its abusing the fact that netboot puts the entire rootfs into the initrd
<clever> tobiasBora: this doesnt actually do any network booting
<clever> tobiasBora: and then line 22/23 will copy the kernel+initrd into /boot, and 17 will load it, obeying the kernel params
<clever> tobiasBora: lines 3-13 will do a nixos eval that imports the netboot modules
<clever> tobiasBora: you may be better off using the netboot stuff
<clever> tobiasBora: the path to fsinfo is then embeded into the script that ultimately lands at /init, and bash will then read it and mount each fs
<clever> tobiasBora: nixos doesnt use root=, but embeds that into the initrd
<clever> ixxie: so you can just > ~/.ssh/authorized_keys to allow yourself on a new machine
<clever> ixxie: something that is very handy, `curl https://github.com/cleverca22.keys` will output every ssh key you authorized to your github
<clever> tobiasBora: so it will expect a kernels directory on the EFI partition
<clever> tobiasBora: line 3 sets $drive1 to whatever FS C863-2483 matches
<clever> tobiasBora: can you pastebin the entire grub.cfg file?
<clever> wdanilo: nix just cant have that problem (when using source everywhere), because a change of the .h file would trigger a rebuild of anything using it
<clever> wdanilo: for example, if the size of a field in a .h file changes, then the ABI for calling functions in that library will be different, and things break in fun ways if you mix the versions up
<clever> wdanilo: the reason nix rebuilds things is for purity, there may be unknown changes in the .so, that impact things at compile or link time, and just changing the search path can break things
<clever> wdanilo: or for darwin, otool
<clever> wdanilo: you can use the patchelf program to do those changes
<clever> 21686148-6449-6E6F-744E-656564454649 is just the hex for "Hah!IdontNeedEFI"
<clever> ixxie: the bios boot partition one is more fun :P
<clever> the real type code is C12A7328-F81F-11D2-BA4B-00A0C93EC93B
<clever> also, those numbers are just for the menu in fdisk
<clever> its the very first type in the list :P
<clever> 1 EFI System C12A7328-F81F-11D2-BA4B-00A0C93EC93B
<clever> wdanilo: and the exact search path is specific to the versions your using as inputs
<clever> wdanilo: yeah, its mainly the rpath field, which has a list of dirs to search for the libs
<clever> ixxie: shift+pageup works in the bare text console
<clever> wdanilo: due to the lack of a /usr/lib directory, nix needs the path to put the paths like /nix/store/hash-name/lib into the ELF headers
<clever> ixxie: did you check the list of type codes, near the start?
<clever> tobiasBora: yeah, it should be possible to patch plasma, its just that nobody has bothered yet
<clever> adamantium: vim
<clever> though the error isnt clear, and claims the fs is wrong
<clever> and the error even mentions ESP
<clever> ixxie: sda2 must be set to the EFI system partition type
<clever> ixxie: why is sda2 set as a bios boot partition? thats only for legacy
<clever> tobiasBora: the problem is that plasma doesnt expect things to be under a symlinked directory, and is watching for changes on the wrong thing
<clever> ixxie: and also `fdisk -l /dev/sda`
<clever> tobiasBora: you usually need to logout and back in for plasma to detect that, it may also need to be in systemPackages
<clever> wdanilo: yeah
<clever> ixxie: can you pastebin the output of `mount ; df -h` ?
<clever> wdanilo: /proc/self is a linux thing, so something must be broken with the --store based chrooting on darwin
<clever> adamantium: but, if you slowly increase the allocations, the zfs ARC shrinks in response, and it works fine
<clever> adamantium: one thing ive noticed, is that qemu tries to allocate several gig at once, and can hard-fail because my laptop lacks enough swap
<clever> ixxie: and did you mount it to /mnt/boot/ ?
<clever> ixxie: that was the exact error it gave?
<clever> yeah
<clever> tobiasBora: i would go with just plain ext4
<clever> tobiasBora: yeah, that sounds right
<clever> srhb: ah, ive not done any special setting, just zfs create, mkswap, and swapon
<clever> srhb: yikes
<clever> my current machine has 32gig of ram, and 32gig of swap, over 4 swap partitions
<clever> ixxie: and if you decide you dont want swap, comment it out, rebuild-switch, then delete the file, and you get your space back
<clever> ixxie: the above will automatically create a 16gig swap file and activate it