2020-09-21

<clever> matthewcroughan: if you never use with, then nix can find typos in your variable names at parse time, and optimize things a little better
<clever> matthewcroughan: there is a foo in the parent scope, and then more in the next scope up the chain
<clever> matthewcroughan: so if you do `let foo = "foo"; in { key = expr; }`
<clever> matthewcroughan: when parsing and evaling, nix has a scope chain
<clever> matthewcroughan: it lets a set refer to its own attributes
<clever> > (rec { cmakeFlags = "--enableWarpDriveOptimizations"; releaseFlags = "${cmakeFlags} --Wall"; }).releaseFlags
<clever> > { cmakeFlags = "--enableWarpDriveOptimizations"; releaseFlags = "${cmakeFlags} --Wall"; } .releaseFlags
<clever> but only the amount of whitespace that is common to each
<clever> on each line
<clever> leading whitespace
<clever> it lets you indent the nix code, without the resulting config file having extra indent
<clever> good
<clever> matthewcroughan: if you make a ''string'' with multiple lines, and 4 spaces at the start of each, all of them are removed
<clever> > let name = "clever"; in "hello ${name}"
<clever> Ke: which is normally managed with nix-channel
<clever> Ke: nope, channels are in a seperate channels profile
<clever> from the man page for nix-env
<clever> $ nix-env -e '.*' (remove everything)
<clever> then only configuration.nix will matter
<clever> repeat removing things until `nix-env -q` and `sudo nix-env -q` are both blank
<clever> Ke: simplest answer, just use `nix-env -e` to uninstall each thing in `nix-env -q`
<clever> and then it replaces the entire rpi3-netboot profile with whatever it just built
<clever> it then loads not-os/release.nix, while overriding what <nixpkgs> maps to, and fetches the rpi_image attribute of that file
<clever> this creates a custom profile called rpi3-netboot
<clever> --set is the declarative mode, used by nixos mainly, but this is an example of using it manually
<clever> nix-env -p /nix/var/nix/profiles/per-user/root/rpi3-netboot -f not-os/release.nix -A rpi_image -I nixpkgs=./nixpkgs/ --set
<clever> nix-env has 2 main modes it can operate in, declarative and imperative mode
<clever> /nix/var/nix/profiles/system is the system profile, managed by configuration.nix
<clever> yeah
<clever> other stuff, it depends
<clever> if you use nix-env -i on /nix/var/nix/profiles/system, yes
<clever> only root can modify root and system, and those are also common to every user
<clever> $USER, root, and system
<clever> there are 3 profiles that executables can come from for each user
<clever> so when people come from debian, and expect root installing something to be global, it is
<clever> root's profile is special, and is in the $PATH for all users
<clever> you should see emacs in `sudo nix-env -q`
<clever> yep, thats roots profile
<clever> do `which emacs` and you should see roots profile
<clever> roots profile acts as a default for all users
<clever> matthewcroughan: tools like home-manager can help to enforce that
<clever> matthewcroughan: then stop using `nix-env -i`
<clever> matthewcroughan: each user, including root, has its own personal profile, none of them have any impact on the system profiles
<clever> matthewcroughan: that would be roots ~/.nix-profile, stuff root installed with nix-env -i
<clever> matthewcroughan: so you can use nix-env without being on nixos (it even works on darwin)
<clever> matthewcroughan: so you can use nix-env without being an admin
<clever> matthewcroughan: no, nix-env only manages ~/.nix-profile on one user
<clever> matthewcroughan: your thinking of `nix-env --list-generations --profile /nix/var/nix/profiles/system`
<clever> matthewcroughan: `nix-env --list-generations` shows generations for your user profile, not the system profile
<clever> matthewcroughan: the pills are now on nixos.org
<clever> matthewcroughan: i learned nix before the pills existed
<clever> but its now got quotes inside a quoted string, and it gets ugly
<clever> nix can even convert a set into json
<clever> > builtins.toJSON({ cmakeFlags = "bananas"; })
<clever> matthewcroughan: let/in let you reuse a value multiple times in a later expr, or give it a name and shove it somewhere out of the way
<clever> matthewcroughan: it overrides src and cmakeFlags

2020-09-20

<clever> sterni: nix path-info can also query it
<clever> 1600578260
<clever> [root@amd-nixos:~]# nix path-info --json /run/current-system | jq .[0].registrationTime
<clever> sterni: its in db.sqlite

2020-09-19

<clever> or just edit configuration.nix instead, since nixos merges them together for you
<clever> colecf: so if you want changes in hardware-configuration.nix to stick, you have to stop running nixos-generate-config
<clever> colecf: nixos-generate-config will scan things like what is currently mounted, and then just blow away the whole hardware-configuration.nix and replace it
<clever> colecf: add it to configuration.nix instead
<clever> it needs its own syntax engine, vim-nix is one of them
<clever> yep
<clever> also, you dont want ${rcfile} but just rcfile;
<clever> its much more visible when gist does syntax highlighting
<clever> sxiii: yeah, the ; is missing on the rc.xml line
<clever> the double autostart just made it look confusing
<clever> thats a variable, from the let block
<clever> oh wait no
<clever> the 2nd autostart also has to be quoted
<clever> environment.etc."openbox/autostart".source = writeScript "autostart" autostart;
<clever> sxiii: the line before openbox/rc.xml, you have an autostart outside of the string
<clever> sxiii: can you pastebin the configuration.nix file?
<clever> la-jesystani: use an overlay, also in the nixpkgs manual
<clever> hyper_ch: if you can configure the VM to forward trim from guest->host, then you can run fstrim inside the guest, or just mount it with auto-trim
<clever> hyper_ch: the zvol has to be mounted only on the host for you to fstrim it
<clever> '
<clever> la-jesystani: nix-build -E 'with import <nixpkgs> {}; callPackage ./. {}
<clever> la-jesystani: nix-build can build it
<clever> hyper_ch: the manual trim tells zfs to forget about the unused blocks, and trim forwarding in the vm would automate it
<clever> hyper_ch: nfs would let zfs know exactly what is happening, so the problem just vanishes
<clever> hyper_ch: so the snapshots are holding onto every single deleted file as well
<clever> hyper_ch: basically, when you delete a file in the guest, zfs doesnt know those blocks are unused
<clever> hyper_ch: but you would still need to repeat that dance regularly, or setup trim forwarding in the vm, or use nfs
<clever> hyper_ch: that will reduce your usage by 307gig immediately, and make any more snapshots you create a bit more optimal
<clever> hyper_ch: umount it within the vm (or shutdown), then mount it anywhere on the host, and run fstrim against it
<clever> hyper_ch: so if you run `blkid /dev/tankHB/encZFS/VMs/Mail` it reports ext4 directly?
<clever> hyper_ch: what is within tankHB/encZFS/VMs/Mail? mbr? ext(2|3|4) ?
<clever> hyper_ch: and its keeping that extra 307gig in every snapshot
<clever> hyper_ch: yeah, you definitely want some trim, your only using 67gig, but zfs is keeping the unused blocks, so its using a total of 374gig for the current version
<clever> la-jesystani: that should be in the nixpkgs manual
<clever> hyper_ch: can you pastebin this? zfs list -t volume -o name,used,referenced,logicalused,logicalreferenced,written,usedbysnapshots,usedbydataset,refcompressratio,compressratio,compression,mountpoint,sync
<clever> hyper_ch: is that a snapshot of a zvol?
<clever> hyper_ch: so if you can configure the VM to pass the trim commands thru, it would help
<clever> hyper_ch: one thing ive found, is that if i mount an ext4 zvol on the zfs host, and run fstrim, the zvol shrinks massively
<clever> hyper_ch: unused blocks in the zvol probably
<clever> hyper_ch: ahh, a zfs filesystem is better for that
<clever> hyper_ch: you can also use a zvol to give the VM its own dedicated disk as a dataset
<clever> hyper_ch: correct, i'm not sure if `zfs nfs export` even works on linux, it might be a solaris-only thing
<clever> hyper_ch: but the clients are just more nixos machines on the LAN, not VM's
<clever> hyper_ch: nfs.server = { enable = true; exports = ''.....''; }; is what ive been using
<clever> and also to just search packages
<clever> i use it to both experiment with new exprs, and to verify a nix file is doing the right thing
<clever> yep
<clever> and just a bit of tab-completing in `nix repl '<nixpkgs>'` and i find this
<clever> > lua52Packages.buildLuarocksPackage
<clever> pkgs/development/interpreters/lua-5/interpreter.nix:luaPackages = callPackage ../../lua-modules {lua=self; overrides=packageOverrides;};
<clever> and now where is lua-modules referenced?
<clever> and which file is that reference in?
<clever> hart111: search nixpkgs to find out where lua-packages.nix gets referenced, what do you find?
<clever> Miyu-saki: then jq to select it out
<clever> Miyu-saki: oh, and nixops export dumps the full state including privkey as json
<clever> Miyu-saki: you can maybe also steal the privkey from that path, while the ssh session is open
<clever> Miyu-saki: ssh -v, to see what pubkeys it tried
<clever> Miyu-saki: dont remember off-hand what the DO backend does with its own keys
<clever> matthewcroughan: cut a few digits off by accident
<clever> Miyu-saki: depending on which backend your using, the nixops key itself may also be in /root/.ssh/authorized_keys
<clever> matthewcroughan: that will install dolphin from master, but every time you nixos-rebuild, it will check if master has changed, and may cause a surprise rebuild if master has changed
<clever> matthewcroughan: https://gist.github.com/cleverca22/5c0d36889c72e1da61cfac42c3c64 and then do `imports = [ ./dolphin-module.nix ];` in configuration.nix
<clever> Miyu-saki: if you have your own keys in /root/.ssh/authorized_keys then you can get in anyways
<clever> matthewcroughan: youll also want an overlay then, to be able to get dolphin master from configuration.nix
<clever> matthewcroughan: not even files in / will persist
<clever> matthewcroughan: thats more extreme, it wipes everything
<clever> matthewcroughan: then use nix-env -e to uninstall things, and `nix-env -q` to list them
<clever> matthewcroughan: nix-shell based stuff wont persist across a reboot
<clever> kini: and the override can only update that if it changes all of the cmakeFlags
<clever> kini: because the original expression is using cmakeFlags to set the version based on the original git rev
<clever> drakonis: the developers encourage you to constantly update to latest, and the emulator itself enforces that everybody in multiplayer has a matching gitrev
<clever> matthewcroughan: nix disables the network, and hashes every single input, so if any input changes, nix redoes the build
<clever> matthewcroughan: it solves the nasty problem of things like `docker build` can just grab whatever it wants over the network, and you get a different version every time you build
<clever> but your changing the cmakeFlags input
<clever> matthewcroughan: if the inputs to a derivation havent changed, nix can reuse the entire output, as a cached build
<clever> matthewcroughan: having something like ccache involved would poison it all
<clever> matthewcroughan: nix's purity is based on hashing every input to a build, and redoing the build if any input has changed
<clever> matthewcroughan: nix only has derivation level caching
<clever> matthewcroughan: correct
<clever> kini: yeah, that would make the override much simpler then what i gave above
<clever> so you must set that to the revision your actually building from
<clever> and if that string doesnt match, it assumes your not compatible
<clever> internally, dolphin bakes DOLPHIN_WC_REVISION into its binary, and then shares that with the other multi-player members
<clever> its the DDOLPHIN_WC_REVISION=
<clever> the version= isnt the problem
<clever> nix-shell -p 'dolphinEmuMaster.overrideAttrs (old: rec { src = builtins.fetchGit "https://github.com/dolphin-emu/dolphin"; version = "dolphin-really-master"; cmakeFlags = [ "-DUSE_SHARED_ENET=ON" "-DENABLE_LTO=ON" "-DDOLPHIN_WC_REVISION=${src.rev}" "-DDOLPHIN_WC_DESCRIBE=really-master" "-DDOLPHIN_WC_BRANCH=master" ]; })'
<clever> one second
<clever> nix fetched, when you used builtins.fetchGit
<clever> and dolphin then shares that with the multi-player members
<clever> that revision, must match the actual git revision you fetched
<clever> "-DDOLPHIN_WC_REVISION=really-master"
<clever> ah, maybe its that...
<clever> because you ran the command at different times, and somebody pushed between the 2 of you starting
<clever> if you use master directly, you run the risk of somebody being 5 minutes newer then the others
<clever> as i said
<clever> same as any other linux distro
<clever> you have to unplug and replug the usb device, for the udev rule to apply
<clever> you also need to fill in the correct vendor and product id
<clever> it must be ''double single quoted''
<clever> the quotes are missing
<clever> services.udev.extraRules = ''ACTION=="add|change", SUBSYSTEM=="usb", ATTRS{idVendor}=="1050", ATTRS{idProduct}=="0407", RUN+="${clearYubikeyUser}"'';
<clever> what group owns that?
<clever> matthewcroughan: ls -l /dev/bus/usb/001/008
<clever> strace -f -e open,openat dolphin-emu
<clever> but dmesg doesnt say what dolphin tried to actually open
<clever> but dmesg doesnt say what dolphin tried to actually open
<clever> matthewcroughan: what device is it trying to open? strace -f -e open,openat dolphin-emu
<clever> matthewcroughan: if master hasnt changed, it will reuse the old build
<clever> matthewcroughan: if master has changed, yes
<clever> matthewcroughan: you have to logout and login for any group changes to take effect, same as every other linux distro
<clever> matthewcroughan: what does `echo $NIX_PATH` say?
<clever> matthewcroughan: that says that you last updated your channels 12 days ago
<clever> matthewcroughan: what does this say?
<clever> [root@system76:~]# nix-env --list-generations --profile /nix/var/nix/profiles/per-user/root/channels
<clever> matthewcroughan: one sec
<clever> matthewcroughan: did you change something involving that?
<clever> copy/paste one line
<clever> what does it say?
<clever> is it actually building them, or just listing paths?
<clever> such as?
<clever> then add it
<clever> matthewcroughan: users.extraUsers.clever.extraGroups = [ "input" ];
<clever> matthewcroughan: you must change groups with configuration.nix
<clever> energizer: --show-trace
<clever> you must be in the input group to open things in /dev/input/
<clever> which device is it failing to access?
<clever> it will build dolphin from master, and put it into the $PATH of a shell
<clever> matthewcroughan: if you changed the cmakeFlags, then it has to rebuild the whole thing
<clever> and it wont be cached, because your making changed hydra hasnt pre-built
<clever> the expression already said to use multiple cores for building
<clever> 33 enableParallelBuilding = true;
<clever> 2020-09-18 22:49:36 < clever> it gets much simpler if you use a config.nix or overlays
<clever> you have to reproduce the cmakeFlags if you want to override it
<clever> nix-shell -p 'dolphinEmuMaster.overrideAttrs (old: { src = builtins.fetchGit "https://github.com/dolphin-emu/dolphin"; version = "dolphin-really-master"; cmakeFlags = [ "-DUSE_SHARED_ENET=ON" "-DENABLE_LTO=ON" "-DDOLPHIN_WC_REVISION=really-master" "-DDOLPHIN_WC_DESCRIBE=really-master" "-DDOLPHIN_WC_BRANCH=master" ]; })'
<clever> the old version# is passed into cmake, and it isnt easily updated
<clever> what i just said
<clever> 52 "-DDOLPHIN_WC_DESCRIBE=${version}"
<clever> but due to how the cmakeFlags are set, its not easy to update the version that gets embedded into the binary
<clever> that will update the name as well
<clever> nix-shell -p 'dolphinEmuMaster.overrideAttrs (old: { src = builtins.fetchGit "https://github.com/dolphin-emu/dolphin"; name = "dolphin-really-master"; })'
<clever> 4*
<clever> matthewcroughan: so its really master, but it claims to be 11825
<clever> matthewcroughan: that command doesnt change the version in the name
<clever> matthewcroughan: what command did you run?
<clever> energizer: its possible that the build fails, so hydra cant cache it
<clever> energizer: does the build pass or fail?
<clever> matthewcroughan: dolphinEmuMaster likely was master at one time
<clever> matthewcroughan: they are using different versions
<clever> 2020-09-18 22:26:57 < clever> dolphinEmu just uses a branch called "5.0" while dolphinEmuMaster uses 1b97f081b8eff9012132a4124537968bdb0e03e0
<clever> so once you exit the shell, its basically gone
<clever> matthewcroughan: this would use nix-shell to shove a build of master into $PATH, without ever installing it
<clever> nix-shell -p 'dolphinEmuMaster.overrideAttrs (old: { src = builtins.fetchGit "https://github.com/dolphin-emu/dolphin"; })'
<clever> it gets much simpler if you use a config.nix or overlays
<clever> matthewcroughan: nix-env -i -E '_: with import <nixpkgs> {}; dolphinEmuMaster.overrideAttrs (old: { src = builtins.fetchGit "https://github.com/dolphin-emu/dolphin"; })'
<clever> matthewcroughan: nix-build only builds, it does not install
<clever> matthewcroughan: you use nix-env instead
<clever> you want result/bin/something
<clever> no amount of logging out will make it appear in $PATH with only nix-build
<clever> matthewcroughan: nix-build doesnt add it to $PATH
<clever> matthewcroughan: did you use the copy of dolphin in the result dir?
<clever> matthewcroughan: did it fail? what was the error?
<clever> matthewcroughan: so you can both run the same version, without the risk of one of you being 5 minutes newer
<clever> matthewcroughan: and thats where reproducibility comes into advantage
<clever> matthewcroughan: if you want multi-player, i think its more important that you both run the same version
<clever> the whole point of nix, is to make things predictable
<clever> because thats not pure, and youll get a different thing every time you do it
<clever> this will fetch the default branch from github, and built it
<clever> nix-build -E 'with import <nixpkgs> {}; dolphinEmuMaster.overrideAttrs (old: { src = builtins.fetchGit "https://github.com/dolphin-emu/dolphin"; })'
<clever> 2020-09-18 22:26:20 < clever> you would need to update the nix expr on every push, or use the import builtins.fetchGit
<clever> then you want builtins.fetchGit instead
<clever> `(old: { src = ./.; })` is a function, that takes anything, and then returns a set containing one key/valye pair, src and ./.
<clever> in this case, a set containing just src
<clever> it takes a set containing the old attrs, and returns something else
<clever> `old: ...` is a function
<clever> it will use the src in .
<clever> so it wont use the fetchFromGitHub in the nix expr
<clever> yeah, it sets the src to that directory
<clever> matthewcroughan: build dolphin
<clever> matthewcroughan: yep
<clever> yeah
<clever> just clone, and run it
<clever> matthewcroughan: that above line will build it for you
<clever> matthewcroughan: clone dolphin, then run that, and it will build it from the current dir
<clever> nix-build -E 'with import <nixpkgs> {}; dolphinEmuMaster.overrideAttrs (old: { src = ./.; })'
<clever> matthewcroughan: you still want nix to compile it, `make install` is just wrong on nix
<clever> or that
<clever> ,tofu matthewcroughan
<clever> matthewcroughan: nix-prefetch-github
<clever> matthewcroughan: you need both the sha1 of a commit, and the sha256 of its contents
<clever> matthewcroughan: you also have to update the rev as well
<clever> `nix edit -f '<nixpkgs>' dolphinEmu` vs `nix edit -f '<nixpkgs>' dolphinEmuMaster` to see the difference
<clever> dolphinEmu just uses a branch called "5.0" while dolphinEmuMaster uses 1b97f081b8eff9012132a4124537968bdb0e03e0
<clever> you would need to update the nix expr on every push, or use the import builtins.fetchGit
<clever> matthewcroughan: because nix wants to know the hash of the source
<clever> matthewcroughan: nothing in nix can ever track master fully
<clever> matthewcroughan: so the -A nixos.dolphinEmuMaster goes into the channel called nixos, then gets .dolphinEmuMaster from it
<clever> matthewcroughan: that creates a set containing each channel
<clever> matthewcroughan: nix-env loads the default expression in ~/.nix-defexpr/ by default
<clever> matthewcroughan: nix-shell -p dolphinEmuMaster
<clever> matthewcroughan: nix-env -iA nixos.dolphinEmuMaster

2020-09-18

<clever> zecnate: you can use `--option sandbox-paths "paths"` to set it manually on a per-build basis
<clever> zecnate: you could set sandbox-paths in nix.conf, to add usr/bin/env
<clever> zecnate: because using /usr/bin/env in the final scripts installed to $out isnt pure, it encourages you to patchShebangs, so it becomes an absolute path
<clever> yep
<clever> yeah
<clever> buckley310: exactly the same way -I works in gcc
<clever> buckley310: it didnt find /var/empty/foo, so it tried the next sconfig in the search path
<clever> buckley310: when you do <sconfig/foo> it will search for a foo in each sconfig
<clever> then just use -I sconfig=
<clever> buckley310: if you want to make smaller changes to just a part of it, the code has to be designed to allow those changes without replacing the whole <sconfig>
<clever> buckley310: `-I sconfig=/path/to` lets you just override the whole thing by force, even if it wasnt made to accept it
<clever> buckley310: all of your nixos modules get merged together, and you can use mkForce to override things
<clever> buckley310: also, anything you put into imports, is a nixos module, which is seperate from nixpkgs
<clever> buckley310: you can only add overrides if it was designed to accept overrides
<clever> buckley310: which other channel do you want to override, and what?
<clever> buckley310: -I nixpkgs=/path/to/nixpkgs
<clever> and you should see the source
<clever> run `nix-shell -p` then `type patchShebangs`
<clever> or to give it a pre-built openssl?
<clever> is there no way to tell it to just do the untar step?
<clever> that maps /bin/sh to busybox when inside the sandbox
<clever> sandbox-paths = /bin/sh=/nix/store/jl42jixjldpvnhw7kgbzxjd2mal6agyw-busybox-1.31.1-x86_64-unknown-linux-musl/bin/busybox
<clever> $ nix show-config | grep sandbox-paths
<clever> zecnate: you must run patchShebangs on the directory where configure lives
<clever> zecnate: /usr/bin/env isnt in the nix sandbox
<clever> jlv1: but you can cheat with (import <nixpkgs>{}).fetchFromGitLab to get your own pkgs
<clever> jlv1: nothing in imports can depend on the pkgs you receive as an input

2020-09-16

<clever> and it queries the version
<clever> that wont let you build stuff that is known-broken
<clever> python has a broken = something; field on each package
<clever> callPackage is also what lets python do that
<clever> then you dont need overlays
<clever> so you have python2Packages.foo and python3Packages.fo
<clever> just put all stuff in a dedicated file, and then callPackage it for every version at once!
<clever> python and kernel modules have a solution to that mess
<clever> jlv1: then you just need the path to the default.nix
<clever> jlv1: super.qt515.callPackage will pass things from qt515 in for you
<clever> zanc: the same set your acting on when you `nix-env -iA nixpkgs.hello`