2019-11-28

<clever> Jarva: you want --set rather then -i
<clever> Jarva: you need to use something like `nix-build '<nixpkgs/nixos>' -A system -I nixos-config=./configuration.nix`
<clever> Jarva: try setting `boot.loader.grub.efiInstallAsRemovable = true;`, rebuild the nixos image, and run the new switch-to-configuration boot
<clever> Jarva: efi or legacy booting?
<clever> Jarva: you need to run the `switch-to-configuration boot` command, to make nixos update /boot fully
<clever> typetetris: you can also `nix show-config | grep sub` to read the current value
<clever> it will overrirde everything
<clever> typetetris: it should work exactly like that
<clever> philipp[m]: you appear to be on irc to me
<clever> pbb: zfs just tries to import in a loop until it works or hits a timeout
<clever> pbb: but btrfs is complicated by needing multiple devices
<clever> pbb: the normal fs stuff, will loop until the device exists
<clever> pbb: yeah, sounds like it needs a bug report
<clever> pbb: try adding a sleep to your initrd script?
<clever> pbb: it might be a timing thing, you type slower, so the devices appeared
<clever> fetchTarball is impure, and can work without a sha256
<clever> that is a valid expression
<clever> > builtins.fetchTarball channel:nixos-unstable
<clever> notgne2: builtins.fetchTarball doesnt need a sha256, and you can use branch names in place of revs, there is also the channel: alias
<clever> its fixed in master, and a work-around is in the PR
<clever> https://github.com/NixOS/nixpkgs/pull/73299 also, wpa_supplicant fails to auto-detect your wifi card, if bonding is loaded at all
<clever> somebody teach a hotel how to do proper wpa auth!
<clever> bloody open wifi and a captive portal
<clever> look at the mess hotels have, lol
<clever> but nobody uses it
<clever> thats the best way to setup wifi where you can disallow one user without having to tell everybody the new pw, on a daily basis
<clever> eyJhb: not really bad, just rare
<clever> eyJhb: i know one uni, that used full WPA-EAP/RSN for auth, you had to use a username and password to connect to the wifi!
<clever> you can likely just drop the mac stuff and it will still work
<clever> but that caused my IP to change, and i didnt have access to the router at the time to fix dhcp config
<clever> i think the wired
<clever> if you dont force the mac's, then bonding picks the mac of one of the IF's
<clever> and if i plug the ethernet back in, the speed goes up
<clever> notgne2: so i can just unplug the ethernet, and it will seamlessly switch to wifi, without changing IP's or dropping tcp conns
<clever> notgne2: this will link the wifi and wired interfaces, and configure it to use whichever is up, but prefer the wired, and use the same mac for both
<clever> notgne2: something ive been playing with lately is bonding, to deal with handover
<clever> notgne2: id say ditch network manager as fast as you can :P
<clever> notgne2: it probably auto-detects all interfaces, i just use plain dhcpcd and wpa_supplicant
<clever> pbb: the initrd may have a more limited udev that cant create it, can you configure btrfs to just look in /dev/ ?
<clever> pbb: i think ive only seen that shortly after re-formatting, udev only creates the symlinks when the device is first inserted
<clever> oh, thats why its not in the docs!
<clever> but i dont see it in the nixos docs...
<clever> coderobe: and may be of use to you, if you want state in /root to persist
<clever> users.users.root.symlinks, oh, thats new to me
<clever> eyJhb: exactly
<clever> xwvvvvwx: i hope thats a OTP
<clever> eyJhb: but if you just shove a quoted path in, that path will stay as-is, and it will symlink something impure!
<clever> eyJhb: i think the way that works, is your supposed to source = ./foo.txt, nix will then copy that to the store, and replace it with "/nix/store/hash-foo.txt", and then nixos will make a symlink to the store
<clever> eyJhb: ah, thats even simpler then i thought
<clever> it cant depend on itself
<clever> need to rename one of them
<clever> eyJhb: withOCC depends on withOCC
<clever> withOCC = ((withOCC || oceSupport) && stdenv.isAarch64);
<clever> gchristensen's variant is using a normal zfs fs for /, but wipes it on bootup, so no ram costs
<clever> hmmm, maybe a systemd prestart or activationScript, if that cant
<clever> coderobe: i think you can use environment.etc to generate a symlink to a more persistent dir
<clever> oceSupport = ! stdenv.isAarch64
<clever> && already returns a bool
<clever> let withOCC = (oceSupport || withOCC) && stdenv.isAarch64;
<clever> evils: they take the form of `if condition then expr1 else expr2`
<clever> evils: ah, if statements dont work like that
<clever> > lib.optional false 42
<clever> > lib.optional true 42
<clever> coderobe: things like postgresql and such
<clever> evils: just use an if statement to conditionally add it to buildInputs, or lib.optional
<clever> coderobe: that reminds me, be aware of any services storing data to /var and such
<clever> evils: anything present in buildInputs is required to build first, so just having it there and not using will still depend on it
<clever> coderobe: environment.etc is used to generate all other files in /etc/
<clever> notgne2: and users.users.foo.initialHashedPassword can be used to deal with things being missing on that first generation
<clever> or use nixops to build it remotely
<clever> and /etc/nixos is optional, you can use -I nixos-config=/path/to/configuration.nix when you nixos-rebuild, to make it load a different path
<clever> cant think of any others at the moment
<clever> nowhere for the journal to go, so when things do fail, you wont have logs of why
<clever> coderobe: and you will want to set uid's for all users you define like that, or they will randomly change, causing you to not own your own home
<clever> coderobe: that defines the pw hash to use, when /etc/passwd was missing
<clever> coderobe: users.users.foo.initialHashedPassword
<clever> ,libraries xel
<clever> xel: your shell.nix can generate a string for you
<clever> > lib.makeLibraryPath [ zlib ncurses ]
<clever> yep, /nix/store is missing
<clever> elvishjerricco: but on the other hand, that would open up things like `systemd-analyze blame` working at the initrd level
<clever> elvishjerricco: i'm a bit torn, on the one hand, our existing initrd works fine, and systemd may get in the way of some things, and just ugh more systemd infestations! lol
<clever> xel: try adding the directory for libX11.so to LD_LIBRARY_PATH ?
<clever> and if things work, then look into patching systemd better
<clever> for testing, you could just disable nuking references and let it bloat a bit
<clever> yeah, thats a bit tricky
<clever> ah
<clever> elvishjerricco: what binary is it? and if you dont patch, it should still work
<clever> elvishjerricco: since it heavily uses offsets to point to things, and shrinking something will move everything after it
<clever> elvishjerricco: the length of strings within the ELF file must not change

2019-11-27

<clever> ?
<clever> dang, and only after the order ships, do i think to add flux to it, lol
<clever> yeah
<clever> once i confirm the boot rom works with larger chips, we could move ahead
<clever> i'm not sure what package i need yet, nor if the boot rom will even allow a bigger spi chip
<clever> so you can force it into MSD mode, on any model without a hub in the way
<clever> with the more capable bootcode.bin (since fixed), you can now configure any gpio pin, to override the name of start.elf
<clever> and with the msg/start.elf from https://github.com/raspberrypi/usbboot, the rpi can emulate a usb stick, and expose its SD card
<clever> coderobe: so you can use usb gadgets in linux to emulate any usb device
<clever> coderobe: oh, another useful thing to note, the usb-c port is wired to the otg usb controller in the cpu
<clever> according to the readme, it can boot linux, but has no real gpu features
<clever> coderobe: one of the next big projects would be getting https://github.com/christinaa/rpi-open-firmware to build with nix
<clever> L and Wb
<clever> samueldr: aha, a table with dimensions!
<clever> samueldr: ah, right shape, wrong size
<clever> but that hello-world doesnt even bring the dram online, so your limited to 128kbyte of ram
<clever> if you nix-build this expr, with the latest nixpkgs master, you will get a working hello world in bootcode.bin
<clever> so you can now compile your own gpu firmware, from nixpkgs
<clever> but, the gpu cross-compiler did make it into nixpkgs
<clever> the stock spi chip is only 512kbyte
<clever> coderobe: it would need some firmware hacking or help from the foundation
<clever> coderobe: and even if the root.squashfs does fit, the currently available gpu firmare wont look there
<clever> and then filtered on that package type
<clever> i'm not 100% sure its the right one, i just opened random parts until the photo looked right
<clever> so, you need to pay nearly $18k to get one, lol
<clever> secondary issue, is that at least for the part i linked, digikey doesnt sell singles, only lots of 4k
<clever> bottom right corner, just to the left of the usb jacks
<clever> coderobe: see the huge chunk of plastic at the bottom? thats the headphone jack
<clever> and anybody else that wants to repeat what ive done, lol
<clever> https://stuff.samueldr.com/IMG_20191125_224354.jpg the problem, is replacing the chip on the left with a 256mb version
<clever> it could boot, without any SD card...
<clever> thats plenty to hold the entire squashfs for not-os
<clever> and the rpi4 boot firmware is on spi flash
<clever> you can find 256mb spi flash chips...
<clever> i just realized something crazy, with the rpi4 .....
<clever> i once used it as a build machine on my rpi, to build arm stuff for a hydra
<clever> and it can be configured to keep /nix/store writable
<clever> 40mb*
<clever> it compiles down to a 40mhz squashfs, a kernel, and an initrd
<clever> coderobe: basically, i took the bare minimum nixos modules, to make a working linux distro, that is semi-nixos-ish, and then wrote some more to fill in the gaps, and ditch systemd entirely
<clever> there is also not-os
<clever> so there is no way for state to persist!
<clever> it doesnt even mount any disks!
<clever> so if you can get a task done without relying on systemd, you could use this to do everything
<clever> the initrd only has 1 file, the haskell binary, it doesnt even have any dynamic libraries!
<clever> coderobe: this will compile a haskell binary, then generate an initrd with that binary at /init, and boot it under qemu
<clever> coderobe: using the internals of nixos, you can also build pretty crazy things, like haskell-init
<clever> coderobe: thats getting fairly heavily into custom and high level stuff
<clever> so it doesnt have the ram costs of a tmpfs, and still looses state on reboot
<clever> i think gchristensen said he has one of his machines running with zfs for the whole disk, and he specially configured it to delete and remake an empty dataset for / on bootup
<clever> which gives you a file-level dedup
<clever> coderobe: there is `nix-store --optimize`, which will hash every file in the store, and then hardlink them to /nix/store/.links/${HASH}
<clever> coderobe: not aware of any special features it uses on any fs
<clever> but you can change those limits as well
<clever> so the journal will be capped to 15% of 50% of your ram
<clever> and a tmpfs defaults to 50% of ram
<clever> it will not use more then 15% of the fs the logs land on
<clever> and scroll down to this section
<clever> SystemMaxUse=, SystemKeepFree=, SystemMaxFileSize=, SystemMaxFiles=, RuntimeMaxUse=, RuntimeKeepFree=, RuntimeMaxFileSize=, RuntimeMaxFiles=
<clever> coderobe: `man journald.conf`
<clever> the journal also gc's itself to keep usage below many thresholds
<clever> yeah, those will take up some space, but not much
<clever> only as much as you write to the fs
<clever> and if the store should be writable, the squashfs is unionfs'd with a tmpfs
<clever> coderobe: all of the installer images mount a squashfs to /nix and a tmpfs to /
<clever> coderobe: nixos can boot with an entirely empty /, even a tmpfs mounted to /, all you need is the store to still be at /nix, and a /boot if you want switch/deploy to still work
<clever> and the journal db goes back weeks
<clever> bennofs: yeah, journald is constantly reading /proc/kmsg (a streaming api to dmesg) and saving them to /var/log/journal/
<clever> bennofs: journalctl -f -t kernel | grep -v refused
<clever> bennofs: you can also just exclude those messages with grep
<clever> ah
<clever> i think its a limitation of iptables, youll need to see if that has a way to log elsewhere
<clever> bennofs: ah, you can also tell it to just not log at all
<clever> bennofs: dmesg does go to the systemd journal, `journalctl -f -t kernel`
<clever> lucasvo: you must `chmod rust_sysroot -R +w` to make it writable, and copy things into it
<clever> lucasvo: line 56 created rust_sysroot, and inherited the read-only from the store
<clever> lovesegfault: previously, it took 4 hours to fail
<clever> its not a -small channel
<clever> ah yeah, forgot about that
<clever> lucasvo: yeah, that could be done
<clever> gchristensen: do you have access to those logs?
<clever> though it has been over 10 since you asked...
<clever> Yaniel: i think its configured to check that tests have passed every 10mins
<clever> lucasvo: but the path may differ if sandboxing is off, you should use $NIX_BUILD_TOP to find it
<clever> lucasvo: /build should always be writable
<clever> meatcar_: for every file in /nix/store/.links/, verify that its hash matches its name
<clever> meatcar_: that does sound like an extra pass verify could do
<clever> but a GC of any amount (nix-collect-garbage --max-freed 1) will clean up the .links
<clever> optimize then re-linked the corrupt one
<clever> so repair without optimize, undid the links, and left a corrupt entry in .links dangling
<clever> meatcar_: and it assumes that hash(${CONTENTS}) == ${HASH}
<clever> meatcar_: nix will hash every file, and hardlink it to /nix/store/.links/${HASH}
<clever> meatcar_: you may need to garbage collect after the first repair
<clever> meatcar_: and the corrupted one went away
<clever> meatcar_: i think what happened, is that `--repair` with the auto-optimize turned off, deleted all corrupt versions of the file, causing the hardlink to be GC'd
<clever> meatcar_: and does verify confirm everything is good now?
<clever> ah
<clever> meatcar_: and then `nix-store -r /nix/store/bdaz46m9h8alxdylwpcbiny3adzh3njb-docbook5-5.0.1 --option auto-optimise-store false` to re-download without optimize
<clever> meatcar_: if its reporting nothing, try `nix-store --delete /nix/store/bdaz46m9h8alxdylwpcbiny3adzh3njb-docbook5-5.0.1`
<clever> meatcar_: without sudo, that makes things worse
<clever> meatcar_: what did `nix-store --query --roots /nix/store/bdaz46m9h8alxdylwpcbiny3adzh3njb-docbook5-5.0.1` report?
<clever> meatcar_: so, nixos-rebuild --option auto-optimise-store false
<clever> auto-optimise-store = false
<clever> $ nix show-config | grep auto
<clever> meatcar_: --option can change those flags
<clever> meatcar_: ah, ive not noticed that before
<clever> meatcar_: nix-store --query --roots /nix/store/bdaz46m9h8alxdylwpcbiny3adzh3njb-docbook5-5.0.1
<clever> otwieracz: but systemd now has some fairly flexible container-like stuff, that operates at a service level
<clever> otwieracz: nixos containers just restart the entire container, rather then reloading systemd within it
<clever> meatcar_: nix-store --repair-path /nix/store/bdaz46m9h8alxdylwpcbiny3adzh3njb-docbook5-5.0.1
<clever> otwieracz: and also point the service itself to /etc/bacular-dir.conf
<clever> meatcar_: nix-store --verify-path /nix/store/bdaz46m9h8alxdylwpcbiny3adzh3njb-docbook5-5.0.1
<clever> meatcar_: try `nixos-rebuild switch -j 1` ?
<clever> meatcar_: try switching to full nixos-unstable, first, and then let me know when it has rebooted into that
<clever> meatcar_: everything it downloads is hashed as it unpacks, and download errors would be detected
<clever> meatcar_: which channel are you on?
<clever> meatcar_: how did it get corrupted?
<clever> philipp[m]: i can see you
<clever> otwieracz: and normally, a change to the .service file triggers a restart, except when reloadIfChanged has been set
<clever> otwieracz: but as a side-effect, the .service file will be changed whenever the restart triggers change
<clever> otwieracz: basically, restartTriggers just adds a field to the .service file, that everything ignores
<clever> otwieracz: i believe so
<clever> otwieracz: /home/clever/apps/nixpkgs/nixos/modules/services/network-filesystems/rsyncd.nix: restartTriggers = [ config.environment.etc."rsyncd.conf".source ];
<clever> otwieracz: and restart triggers so nixos still notices that things have changed
<clever> otwieracz: use environment.etc to put the config into /etc/
<clever> cmacrae: you can also put the same mkIf's in each other file
<clever> cmacrae: its also better to ensure imports is always a path, and to not import within imports, that makes the errors harder to read
<clever> cmacrae: but if your module lacks both config and options, the module system will wrap the whole thing in config = { ... }; for you, and move imports out
<clever> cmacrae: imports is supposed to be a sibling of config, not a child
<clever> pbb: ls -l /dev/sd* ?
<clever> pbb: what does `lsblk` report in the initrd?
<clever> typetetris: nixos normally goes out of its way to not restart the display-manager
<clever> avn: nscd i think is what he meant
<clever> pbb: such as debug1devices and similar, you can find them all in the stage-1-init.sh source
<clever> pbb: there are several flags to force a failure
<clever> and systemd will pass you the MAINPID
<clever> /home/clever/apps/nixpkgs/nixos/modules/services/mail/rmilter.nix: ExecReload = "${pkgs.coreutils}/bin/kill -USR1 $MAINPID";
<clever> you can also tell it how to reload
<clever> /home/clever/apps/nixpkgs/nixos/modules/services/monitoring/monit.nix: ExecReload = "${pkgs.monit}/bin/monit -c /etc/monitrc reload";
<clever> otwieracz: here is a random example
<clever> /home/clever/apps/nixpkgs/nixos/modules/services/network-filesystems/u9fs.nix: reloadIfChanged = true;
<clever> otwieracz: yes
<clever> Ariakenom: i believe nix-shell is parsing and handling the 2nd #!
<clever> jumper149: you could wrapProgram mutt in another derivation, to add urlview to its PATH
<clever> jumper149: lib.optional
<clever> gchristensen: am now

2019-11-26

<clever> boxscape: `nix build -f filename.nix package`
<clever> yep
<clever> m1cr0man: release.nix has always worked like that
<clever> m1cr0man: add .x86_64-linux to the end of the -A
<clever> m1cr0man: the netboot attribute is a set of netboot images for every arch
<clever> but over the weekend, it played musical chairs with my nix store, and got it into its own dataset
<clever> so, nix-collect-garbage was a no-op, lol
<clever> and i like to snapshot /
<clever> DigitalKiwi: when i had first installed my nas, i made the mistake of leaving /nix on the / dataset
<clever> elvishjerricco: flipped over to lz4, and i'll leave it like that for a while
<clever> ah, uncompressed data was added, making the ratio worse
<clever> naspool/nix 31.9G 31.9G 57.9G 57.9G 31.9G 0B 31.9G 2.19x 2.19x gzip-9
<clever> the ratio was higher earlier
<clever> but its still giving a ~20gig savings
<clever> i have since turned compression off, to make future writes faster
<clever> naspool/nix 44.3G 44.3G 65.8G 65.8G 44.3G 0B 44.3G 1.87x 1.87x off
<clever> NAME USED REFER LUSED LREFER WRITTEN USEDSNAP USEDDS REFRATIO RATIO COMPRESS
<clever> [root@nas:~]# zfs list -t filesystem -o name,used,referenced,logicalused,logicalreferenced,written,usedbysnapshots,usedbydataset,refcompressratio,compressratio,compression
<clever> elvishjerricco: i believe it will replace the tail block until it hits some min size, but the blocks will be larger if you wrote more without a sync
<clever> elvishjerricco: it it wont make a ton of 1 byte blocks
<clever> elvishjerricco: i think it will replace the tail block each time, until it hits some min block size
<clever> DigitalKiwi: and i got a 2x ratio after re-writing the entire store back to disk on zfs
<clever> DigitalKiwi: i turned on gzip-9 when i was fixing my nix store on the nas (moving it from / to /nix)
<clever> nDuff: i think its just preInstall, not preInstallPhase
<clever> nDuff: is it even getting to the preInstall? add an echo to it?
<clever> that also sounds simple and easy
<clever> exarkun: yeah, just { nodes, pkgs, config, ... }:
<clever> exarkun: and i think you can get a reference to the other machines, much like nixops
<clever> exarkun: this is how the hosts file is generated
<clever> exarkun: refer to the pr that just got merged above
<clever> exarkun: what about the python test framework?
<clever> ah right
<clever> what stops you from using that name?
<clever> exarkun: if you write a test to cat /etc/hosts, what does it find?
<clever> exarkun: i believe the machines are already in eachothers hosts files
<clever> vaibhavsagar: it may be that zoom-us is broken then
<clever> vaibhavsagar: are there any qt files in ~/.nix-profile/lib/ ?
<clever> dredozubov: `trusted-public-keys` in nix.conf needs to be fixed
<clever> dredozubov: its saying they have a copy, but you dont trust the signature
<clever> dredozubov: you need to fix this first
<clever> warning: substituter 'https://cache.nixos.org' does not have a valid signature for path '/nix/store/m80snlingdn5qgnklzaixscsvw2gsamq-ghc-8.6.5'
<clever> warning: substituter 'ssh://b2builder' does not have a valid signature for path '/nix/store/m80snlingdn5qgnklzaixscsvw2gsamq-ghc-8.6.5'
<clever> dredozubov: can you pastebin the entire output, up to when it started to build
<clever> dredozubov: but if ran on a .drv file, no
<clever> dredozubov: if ran on an output, yes
<clever> dredozubov: and then nix-copy-closure the get the products back
<clever> dredozubov: you can also use `nix-copy-closure` to copy a `.drv` file to a remote machine, and then run `nix-store -r` on the drv to build it